The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents

2026-07-24

AI Summary

This study addresses a critical yet overlooked issue in large language model (LLM) agents: while procedural skills improve average task success rates, they often induce “regression”—causing previously solvable tasks to fail. Through controlled experiments on nearly 6,000 office automation tasks, this work quantifies and disentangles the dual effects of skill integration, introducing the concept of a “regression tax.” The findings reveal that skill reliability hinges more on grounding and verification mechanisms than on procedural logic itself; the superiority of optimal skills stems primarily from their lower regression rates; and most regression failures can be mitigated through enhanced verification. These insights establish a new paradigm for skill design, grounded in empirical evidence and emphasizing robustness over mere capability expansion.

Abstract

Adding procedural skills to an LLM agent is typically evaluated by average improvement in task success. However, this metric hides an important cost: skills can also make agents worse. We measure both sides by comparing agents with and without skills across nearly 6,000 runs spanning two office automation benchmarks and three model harness stacks. This allows us to distinguish two outcomes. A regression is a task solved without skills but failed after skills are added. A residual failure is a task that fails both with and without skills. We find that regressions are substantial enough that the best performing skills outperform others primarily by regressing less, not by gaining more. We identify three causes of regression: (i) skill description osmosis, a skill changes an agent's behavior simply by being present in context, even when it is never invoked; (ii) grounding displacement, a skill's prescribed procedure overrides how the agent interprets its inputs; and (iii) verification displacement, where the procedure suppresses checks the agent would otherwise perform on its outputs. Analysing persistent failures reveals the same underlying pattern. Existing skills overemphasize procedural guidance the stage least often responsible for failure while under supporting grounding and verification, the dominant sources of remaining errors. After correcting evaluation artifacts and studying traces, we find many regressions and persistent failures recoverable through better grounding and verification. Procedural skills should be evaluated by decomposing their net effect into gains and regressions, not by aggregate improvement alone. We identify three regression modes skills should avoid, and find that reliability depends more on grounding and verification than on procedural skill choice.

Problem

• regression
• procedural skills
• LLM agents
• grounding
• verification

Innovation

• regression
• grounding
• verification
• procedural skills
• LLM agents