DC outage Tuesday 23rd July 2024 15:07:28

Hi,

due to issues with our storage backend most public facing services are currently experiencing downtime. We collaborate with SUSE to resolve the situation - as the failure occurred multiple times over the last days with a solution from the vendor still pending, a software downgrade is currently being worked on.

Download services have now been restored as well.

Posted 2 months ago by georg

All openSUSE Heroes operated services have been restored, but minor cleanup work is pending. The teams are still working on restoring download.opensuse.org.

Posted 2 months ago by georg

We recovered our base infrastructure and are now ensuring all services are fully operational again.

Posted 2 months ago by georg

Our storage systems are now fully recovered, and work has started on restoring the various services.

Posted 2 months ago by georg

One of two storage systems was restored, the second one is still pending. Once both systems are back in operation, services will be recovered. Additionally, an engineering recommendation was implemented to reduce the risk of further outages until an official solution from the storage vendor is released.

Posted 2 months ago by georg

Downgrade of problematic software is now in progress.

Posted 2 months ago by georg