Access Issues for Non-Admin Users

Minor incident EU Environment API (EU Environment) Application/UI (EU Environment) US Environment API (US Environment) Application/UI (US Environment)
2025-06-05 08:03 IDT · 36 minutes

Updates

Post-mortem

Summary:

On June 5, 2025, multiple users were unexpectedly logged out after logging into the application. A newly introduced validation rule in the Notifications Center service incorrectly rejected certain requests and returned HTTP 401 for Non-admin users, which the frontend interpreted as a session loss. Rolling back the validator logic immediately restored normal operation.

Timeline:

05.06.25 08:03 PM – Support received reports of session disconnects and display‑name anomalies.

05.06.25 08:04 PM – Engineering acknowledged the incident and began an investigation.

05.06.25 08:25 PM – Investigation traced the issue to a backend validator returning HTTP 401.

05.06.25 08:34 PM – A recent change in the Notifications Center was identified as the root cause.

05.06.25 08:37 PM – Fix prepared, merged, and deployed.

05.06.25 08:39 PM – Access confirmed restored; incident closed.

Root Cause Analysis:

Incorrect validation logic in the Notifications Center rejected requests for non‑admin users. The service responded with HTTP 401 Unauthorized for Non-admin users, causing the frontend to perform a logout.

Action Items:

Improve client‑side error handling to surface precise messages instead of disconnecting on HTTP 401.

Return HTTP 403 for authorization errors.

Update the code‑review checklist to include validation of shared identity contracts.

June 5, 2025 · 08:39 IDT
Issue

This incident has been resolved.

June 5, 2025 · 08:03 IDT

← Back