Currently the web UI component of openqa.opensuse.org often responds with "504 Gateway Time-Out" and is experiencing connectivity problems to its workers. We are investigating the issue which seems to be related to network problems.
Updated from "Partial Outage" to "Performance Issues" as we are back to a mainly stable state which we have under control with a reduced set of active workers.
We will shutdown the machine o3 and bring it up with more cores in the next minutes. o3 is back, picked up jobs, continuing the monitoring.
All problems resolved. Risk mitigation improvements have been implemented or are planned to be implemented in the future.
See https://progress.opensuse.org/issues/39743 for details.