Summary
During the incident, customers experienced delays and temporary disruptions in scans. The issue was caused by a degraded performance of a process responsible for starting the scans. This means that the processing environment was underutilized even though there were enqueued scans. A corrective update was deployed to stabilize the environment, and the system has since returned to expected performance levels.
Key Timeline (IDT)
07.06.2026, 07:00 IDT: Degraded performance of scans began.
08.06.2026, 11:00 IDT: Degraded performance was noted and the investigation began.
08.06.2026, 13:15 IDT: A bottleneck was identified and a corrective update was deployed.
08.06.2026, 14:00 IDT: It was confirmed that the update increased the throughput of the system.
09.06.2026, 14:00 IDT: System stability was confirmed, and the system was restored to normal operation.
Root Cause
The issue was triggered by degraded performance of a process responsible for starting scans. This caused the processing environment to be underutilized (even as low as 30% processing capacity), despite there being enqueued scans.
The root cause of the degraded performance has been identified in a race condition when attempting to rerun a failed scan. When attempting to rescan, the system occasionally encountered an error, because the first attempt of a scan failed, and the job was in terminating state. This resulted in extensive retries of the starting the rescans, which caused the degraded performance.
Actions Taken
Deployed a corrective update to the bottleneck process.
Restored expected processing performance.
Improved monitoring to detect degraded performance.
Action Items
Further improve monitoring to detect degraded performance.
Further improve the performance of the bottleneck process.
Resolved
Summary
During the incident, customers experienced delays and temporary disruptions in scans. The issue was caused by a degraded performance of a process responsible for starting the scans. This means that the processing environment was underutilized even though there were enqueued scans. A corrective update was deployed to stabilize the environment, and the system has since returned to expected performance levels.
Key Timeline (IDT)
07.06.2026, 07:00 IDT: Degraded performance of scans began.
08.06.2026, 11:00 IDT: Degraded performance was noted and the investigation began.
08.06.2026, 13:15 IDT: A bottleneck was identified and a corrective update was deployed.
08.06.2026, 14:00 IDT: It was confirmed that the update increased the throughput of the system.
09.06.2026, 14:00 IDT: System stability was confirmed, and the system was restored to normal operation.
Root Cause
The issue was triggered by degraded performance of a process responsible for starting scans. This caused the processing environment to be underutilized (even as low as 30% processing capacity), despite there being enqueued scans.
The root cause of the degraded performance has been identified in a race condition when attempting to rerun a failed scan. When attempting to rescan, the system occasionally encountered an error, because the first attempt of a scan failed, and the job was in terminating state. This resulted in extensive retries of the starting the rescans, which caused the degraded performance.
Actions Taken
Deployed a corrective update to the bottleneck process.
Restored expected processing performance.
Improved monitoring to detect degraded performance.
Action Items
Further improve monitoring to detect degraded performance.
Further improve the performance of the bottleneck process.
Resolved
System is now fully stable
Monitoring
The scans are now stable and there should be no more degraded performance. The team is continuing to monitor the situation.
Monitoring
We are seeing the system is now more stable. We are monitoring the situation.
Identified
The team has identified the bottleneck and is working on resolving the issue.
Investigating
We have noticed degraded performance in on demand scans.
Pull request and CLI scans should not be affected.
We have seen degraded performance in SAST scans for all customers, and SCA and Secret scans for some customers.
We are monitoring and evaluating the situation