The maintenance has been canceled due to organizational reasons. A new date will be announced separately.
On Friday, 2012-08-24 between 22:00 and 24:00 CEST we will replace a switch which has accumulated defect ports in the last months. We do not expect any services to have any visible interruption due to the change.
The exchange will be performed by adding the new switch and slowly migrating all connections from the old switch to the new one causing only small, intermittent interruptions (time needed to move the cable plus a few seconds for RSTP to enable the port).
Individual affected services should show a temporary increase in response times but no actual failures.
![gocept.net [en]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY65wXXMC4QddrR7HTZnjU9u-41YNivbxQFTE7ptP_w-pxs5E-UwY2lF_zetWI19slZ_BIYljxRVsXdeUffNfpsoyKXOaA-jj45jrcuIAdUZiSTYDyjEAckKVIODgfGWWFqc-JuybhbgRa/s1600-r/logo.png){kind=logo}
Intermittent connection problems with various hosted web sites
Since Monday (2012-07-09), several users report connection problems with web sites hosted at gocept.net. Typical symptoms include painfully slow pages and browser timeouts.
Until now we have a hard time to diagnose the problems as we cannot consistently reproduce them. It looks like only some users and some sites are affected. To get this issue fixed soon, we would appreciate user feedback.
So if you are experiencing problems like intermittent hangs or connection timeouts with gocept.net hosted websites right now, please go to http://supportdetails.com/?recipient=support@gocept.com, please fill in your name and e-mail address and send a report. This would help us greatly to gather relevant data and work towards a solution.
Until now we have a hard time to diagnose the problems as we cannot consistently reproduce them. It looks like only some users and some sites are affected. To get this issue fixed soon, we would appreciate user feedback.
So if you are experiencing problems like intermittent hangs or connection timeouts with gocept.net hosted websites right now, please go to http://supportdetails.com/?recipient=support@gocept.com, please fill in your name and e-mail address and send a report. This would help us greatly to gather relevant data and work towards a solution.
Security update: beware of broken shared library dependencies
Today (2012-06-28) we are rolling out an unusually large security update. The update is necessary to fix known vulnerabilities. Unfortunately, the change has a small chance of breaking customer-compiled C extensions. We did our best to retain binary compatibility but we cannot guarantee it in every single case.
We ask all developers that have installed C extensions to verify that their programs are still starting up correctly. If not, a simple recompile should be sufficient to fix the problem.
We ask all developers that have installed C extensions to verify that their programs are still starting up correctly. If not, a simple recompile should be sufficient to fix the problem.
Maintenance of switching infrastructure on 2012-07-05
On 2012-07-05 between 22:00pm and 23:00pm CEST we will change the power connection of one of our switches. This will cause a short interruption of network connectivity for some VMs. We expect the interruption to take less than 5 minutes.
This change is necessary to modernize the power distribution system within our racks. We try to perform this using redundant power connections without interruption of service as far as possible. During the next weeks we will also update our second rack which will require another short switch interruption. We will announce the required maintenance window separately.
Inconsistent PostgreSQL backups
Yesterday we found and fixed a critical bug in our backup server configuration.
A typo in the exclude file caused PostgreSQL write-ahead logs (WAL) not to be backed up. The PostgreSQL heap, which contains regular table data, has been backed up constantly though. This means that we were able to restore PostgreSQL data, but could not guarantee a consistent restore of transactions started between the last checkpoint and the time the snapshot for the backup was made.
The bug has been fixed yesterday. We have verified that all PostgreSQL servers' write-ahead logs have been backed up tonight so restores from backups performed since 2012-05-16 will not suffer from this problem.
A typo in the exclude file caused PostgreSQL write-ahead logs (WAL) not to be backed up. The PostgreSQL heap, which contains regular table data, has been backed up constantly though. This means that we were able to restore PostgreSQL data, but could not guarantee a consistent restore of transactions started between the last checkpoint and the time the snapshot for the backup was made.
The bug has been fixed yesterday. We have verified that all PostgreSQL servers' write-ahead logs have been backed up tonight so restores from backups performed since 2012-05-16 will not suffer from this problem.
Network maintenance on 2012-04-26 at 21:30-22:30 CEST
We will update our switch firmware on Thursday 2012-04-26 between 21:30 and 22:30 CEST (19:30-20:30 UTC). During these updates, we expect several short losses of network connectivity (<5min). Please note that customer VMs may be unreachable for short amounts of time.
The switch firmware updates will help us to ensure future network reliability and security. We apologize for any inconvenience.
The switch firmware updates will help us to ensure future network reliability and security. We apologize for any inconvenience.
Connectivity Issues 2012-04-10 (12:07pm-12:33pm CEST)
From 12:07pm to 12:33pm CEST we experienced a connectivity loss on our upstream connection to the data center. All physical and virtual customer machines have been affected.
The connectivity loss was caused by faulty switch hardware of our data center operator. The switch has been replaced immediately and connectivity has been restored.
We apologize for the service interruption.
Subscribe to:
Posts (Atom)