Retail Management & Subscription Updates Intermittently Failing

Minor incident Main region Retail services
2025-07-17 00:00 CEST · 10 hours

Updates

Retroactive

What happened?
Between 22:00 UTC (16 Jul) and 08:00 UTC (17 Jul), a minority of requests for users, devices, and subscriptions timed-out. Customers saw failures in the End-Customer Portal and through the Management & Subscription APIs.

Impact
Provisioning, configuration tasks, and data retrieval were unreliable for all customers. Failed actions must be retried.

Resolution
We found the root cause which was related to a degraded message queue cluster and applied a fix, after which error rates rapidly returned to normal by 08:00 UTC.

July 17, 2025 · 09:51 CEST

← Back