Summary
During the incident, customers experienced delays and temporary disruptions in on demand scans. The issue was caused by a number of redundant SCA scans as well as unoptimized reachability scans. As a result, processing capacity was improperly used for redundant, duplicate, scans. A corrective update for both issues was deployed to stabilize the environment, and the processing environment has since returned to expected performance levels.
Key Timeline (IDT)
01.06.2026: Unoptimized code responsible for reachability scans was introduced.
03.06.2026: A faulty code responsible for redundant SCA scans was introduced.
11.06.2026, 16:00 IDT: Increased processing delays were detected, and an investigation into the scanning service began.
11.06.2026, 17:00 IDT: The root cause for redundant SCA scans was identified, and the team began working on improving the behavior.
12.06.2026, 10:30 IDT: The root cause of redundant SCA scans was resolved.
12.06.2026, 17:00 IDT: The root cause for degraded performance in reachability scans was identified.
15.06.2026, 16:00 IDT: A corrective update was deployed to restore the optimization of reachability scans.
15.06.2026, 17:00 IDT: System stability was confirmed.
Root Cause
The issue was triggered by a combination of two issues:
Redundant SCA scans -- Upon receiving a new branch created event from the source code manager, the system attempted to scan all available branches that were configured to be scanned in the repository. In extreme cases, the team has found a magnitude of new scans started instead of one scan, when a branch was created. A faulty code introduced this bug on 03.06.2026 and got resolved on 12.06.2026
Unoptimized reachability scans -- Reachability scanner in the processing environment, has been configured to scan a set of programming languages. However, due to misconfiguration, the scanner was also attempting to scan programming languages that have not been configured to be scanned. Additionally, the scanner has been running on programming languages that are not stable yet, and in extreme cases, it caused a portion of scans to be stuck until they got timed out.
Actions Taken
Deployed a corrective update to remove redundant SCA scans.
Deployed a performance upgrade to reachability scans.
Action Items
Improve monitoring to better understand the behavior of the processing environment.
Improve scanning optimization for all scan types.
Resolved
Summary
During the incident, customers experienced delays and temporary disruptions in on demand scans. The issue was caused by a number of redundant SCA scans as well as unoptimized reachability scans. As a result, processing capacity was improperly used for redundant, duplicate, scans. A corrective update for both issues was deployed to stabilize the environment, and the processing environment has since returned to expected performance levels.
Key Timeline (IDT)
01.06.2026: Unoptimized code responsible for reachability scans was introduced.
03.06.2026: A faulty code responsible for redundant SCA scans was introduced.
11.06.2026, 16:00 IDT: Increased processing delays were detected, and an investigation into the scanning service began.
11.06.2026, 17:00 IDT: The root cause for redundant SCA scans was identified, and the team began working on improving the behavior.
12.06.2026, 10:30 IDT: The root cause of redundant SCA scans was resolved.
12.06.2026, 17:00 IDT: The root cause for degraded performance in reachability scans was identified.
15.06.2026, 16:00 IDT: A corrective update was deployed to restore the optimization of reachability scans.
15.06.2026, 17:00 IDT: System stability was confirmed.
Root Cause
The issue was triggered by a combination of two issues:
Redundant SCA scans -- Upon receiving a new branch created event from the source code manager, the system attempted to scan all available branches that were configured to be scanned in the repository. In extreme cases, the team has found a magnitude of new scans started instead of one scan, when a branch was created. A faulty code introduced this bug on 03.06.2026 and got resolved on 12.06.2026
Unoptimized reachability scans -- Reachability scanner in the processing environment, has been configured to scan a set of programming languages. However, due to misconfiguration, the scanner was also attempting to scan programming languages that have not been configured to be scanned. Additionally, the scanner has been running on programming languages that are not stable yet, and in extreme cases, it caused a portion of scans to be stuck until they got timed out.
Actions Taken
Deployed a corrective update to remove redundant SCA scans.
Deployed a performance upgrade to reachability scans.
Action Items
Improve monitoring to better understand the behavior of the processing environment.
Improve scanning optimization for all scan types.
Resolved
The system is now fully operational. The team is looking into the root cause of the incident.
Monitoring
Most of the scans should now operate with regular performance.
Investigating
We are seeing degraded performance in some on demand scans. Some scans may remain in Running status longer than usual. CLI and PR scans are not affected.