Trajectory-Level Security Debt in LLM Coding Agents

2026-09-28

AI Summary

This study addresses the limitation of existing evaluations that focus solely on final code while overlooking safety risks in the intermediate processes of LLM-based coding agents. To this end, it proposes "Safety Debt Line Integral," a novel trajectory-level metric. By integrating Static Application Security Testing (SAST) with Common Weakness Enumeration (CWE) classification, this method dynamically correlates code evolution with static scanning results to quantify the cumulative risk incurred as agents improve test pass rates. Empirical analyses on benchmarks such as SWE-bench reveal low agreement among multiple detection tools and demonstrate that the proposed metric effectively tracks both development progress and safety debt simultaneously. Ultimately, this work offers a new perspective for the safety-aligned steering of autonomous coding agents.

Abstract

LLM coding agents can traverse hundreds of intermediate code states before submitting a solution. Evaluating only the final artifact leaves the evolution of security findings unmeasured. We introduce the Security Debt Line Integral (SDLI), which accumulates static-analysis risk when an agent reaches a new best test pass ratio. We instantiate it with four static application security testing (SAST) tools and study artifacts from 830 passing SWE-bench runs, 712 ProgramBench final workspaces, and 13 public MirrorCode trajectories. The two large populations use the final-state special case of SDLI. Two-tool Common Weakness Enumeration (CWE) class agreement occurs in 3.9% of SWE-bench runs and 26.2% of the 80 ProgramBench runs passing at least 90% of official tests. These are scanner findings, not validated vulnerability rates. Excluding three advisory-heavy classes reduces the latter rate to 6.2%. Same-task runs differ in their measured scores, while one reconstructed ProgramBench run exposes persistent findings from its first implementation write. A repair case study reduces the scanner signal while preserving tested behavior, but also reveals sensitivity to equivalent API rewrites. SDLI offers a way to study progress and security findings together. Its value for steering agents and confirming exploitable vulnerabilities remains to be established.

Problem

• LLM coding agents
• security debt
• trajectory-level security
• static application security testing
• intermediate code states

Innovation

• Security Debt Line Integral
• LLM Coding Agents
• Trajectory-Level Security
• Static Application Security Testing
• SWE-bench