On Tuesday (2011-10-18) at 22:45 CEST an attacker managed to gain access to one of our internal systems running, among others, the external service monitoring, the Redmine project management tool, and an internal mailing list server. The compromised system is not automatically managed.
Customer VMs were not affected.
We needed to take the machine partially off the network between Wednesday (2011-10-19) 13:20 and 21:00 CEST to perform an analysis and fix security holes. The above mentioned services had only limited availability during this period. User-generated data has most likely not been compromised. We decided to take the machine back online to make the services available again as quickly as possible.
Nevertheless we will move the services to newly installed machines shortly and erase the compromised machine. We will review and improve our security practices to avoid similar incidents in the future.