All customers was affected by a downtime of the system for 9 minutes (11:13 to 11:20) due to an deployment incident. We apologise the problems that this created for some users.

‌

At 11:09 operation (devops) have been contacted by dev team that something is probably not right with an ongoing deployment. That it has stuck on Database migrations step.

We immediately spotted that an deployment was stuck on a step.

By canceling the deployment the system operational status was restored.

Later, the same day, we rerun this migration with no issues.

We suspect that a system related job blocked the change and cause the incident.

As for the impact, it has stalled the db at 11:13 to 11:22 (9 minutes), causing requests to queue and slow down/timeout.

We will follow similar deployments in the future and ensure that the database migration steps are working as they should.