Friday, February 27, 2009
Master Mysql Server
We are investigating an issue causing the main mysql server to reject new connections from the web servers. Further updates will follow once we have identified the cause.
UPDATE
The service has been restarted a few times now but with no apparent resolution to the issue. We are continuing to look for the cause of the issue.
UPDATE
The issue with the mysql server has been tracked back to a faulty connection with one of the 4 dns servers. All services have been restored and are running normally.
UPDATE
The service has been restarted a few times now but with no apparent resolution to the issue. We are continuing to look for the cause of the issue.
UPDATE
The issue with the mysql server has been tracked back to a faulty connection with one of the 4 dns servers. All services have been restored and are running normally.
Monday, February 09, 2009
LVS load balancer rteboot
The LVS load balancer has been rebooted after a crash, the downtime was 14 minutes.
All services appear to be running normally.
All services appear to be running normally.
Tuesday, February 03, 2009
Mail server issues
We are currently investigating two issues with the mail servers.
1) Possible authentication issue for sending mail
2) Failure of the file server holding pop/imap and webmail.
A reboot is being undertaken of the file server and we are tracing the cause of the authentication issue for sending email.
Further updates to follow.
UPDATE
No issues with mail authentication have been found, inbound and outbound mail are working normally.
The pop/imap/webmail file server is having a disk check performed before we bring it live again. That should be completed shortly.
UPDATE
Mail storage services are all restored and working again. We have identified a single domain that has been mailbombed so are taking steps to reduce the impact of the attack on the mail servers.
All customer mail services should be working normally now.
1) Possible authentication issue for sending mail
2) Failure of the file server holding pop/imap and webmail.
A reboot is being undertaken of the file server and we are tracing the cause of the authentication issue for sending email.
Further updates to follow.
UPDATE
No issues with mail authentication have been found, inbound and outbound mail are working normally.
The pop/imap/webmail file server is having a disk check performed before we bring it live again. That should be completed shortly.
UPDATE
Mail storage services are all restored and working again. We have identified a single domain that has been mailbombed so are taking steps to reduce the impact of the attack on the mail servers.
All customer mail services should be working normally now.
