Harness Annealing: Learning to Act with Less External Control

2026-10-01

AI Summary

This study addresses the over-reliance of language agents on external frameworks for state tracking and control decision-making by proposing Harness Annealing Training, a method designed to internalize control capabilities. Introducing the novel concept of harness annealing, this approach integrates explicit control supervision with a progressive attenuation strategy. Through curriculum learning on teacher trajectories, it distills successful trajectories into the model’s intrinsic decision-making capacity while dynamically adjusting the allocation of control authority between human and machine. Experimental results demonstrate that 9B- and 35B-parameter models, when relying solely on tool use, achieve performance comparable to full framework deployment, thereby significantly reducing runtime control overhead.

Abstract

Language agents rely on external harnesses to track state, organize workflows, and verify answers. Beyond providing tools and information, these harnesses supply control decisions about what to investigate, whether to revise, and when to stop. Training on successful harness-supported trajectories can improve task performance while leaving these decisions dependent on runtime intervention. We ask whether harness-supported experience can also teach the model to make these decisions, allowing the division of control to change as the model learns. We call this objective harness internalization: learning to assume specified control responsibilities while retaining task performance after the corresponding support is withdrawn. We introduce HARNESS ANNEALING TRAINING (HAT), which combines explicit control supervision with a curriculum over teacher trajectories collected under progressively weaker harnesses. Experiments with 9B and 35B models on SWE-QA and SWE-QA-Pro evaluate every checkpoint under four deployment harnesses. Selected annealed checkpoints operating with tools alone achieve scores close to those of their respective starting checkpoints deployed with the full harness. The benefits vary with model scale and deployment configuration, and further annealing does not uniformly improve performance. These findings suggest that harness-supported experience can help reduce the runtime control required by a trained agent.

Problem

• language agents
• harness internalization
• external control
• control decisions
• autonomous decision-making

Innovation

• Harness Annealing
• Control Internalization
• Curriculum Learning
• Language Agents
• Explicit Control Supervision