Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

2026-08-03

AI Summary

This work addresses the challenge that large language model agents struggle to dynamically refine their runtime execution frameworks using failure trajectories during deployment. To overcome this limitation, the authors propose training a dedicated “harness engineer” model that performs end-to-end, failure-conditioned editing of executable harnesses throughout their lifecycle. This is achieved through an initial cold-start supervised fine-tuning phase followed by online reinforcement learning via group relative policy optimization. The approach uniquely formulates harness editing as a learnable behavior and enables co-evolution with the target agent. Experimental results demonstrate that, on WebShop, ALFWorld, and DBBench, the method improves the base task success rate of Qwen3.5-9B from 44.3% to 53.6%, and further to 64.2% when combined with agent fine-tuning.

Abstract

Agents built around large language models continually accumulate interaction trajectories during deployment, yet their behavior typically remains fixed. Beyond updating model weights, these trajectories can improve the agent harness that constructs context, mediates tools, validates actions, and recovers execution. We introduce Harness-R1, the first method, to our knowledge, that makes failure-conditioned, lifecycle-wide editing of an existing executable runtime a learned capability. It post-trains a dedicated harness engineer with online reinforcement learning so that its edits are optimized for the realized task success they produce, rather than proposed by a fixed editor. A separate 9B engineer converts batches of target-agent failures into validated executable patches; fresh same-batch reruns of the frozen target provide outcome rewards, so training updates only the engineer. Cold-start supervised fine-tuning initializes this editing policy, which is then trained online with group-relative policy optimization. Across WebShop, ALFWorld, and DBBench, Harness-R1 raises vanilla Qwen3.5-9B success from 44.3% to 53.6% (+9.3 percentage points). After direct target-agent fine-tuning, a target-specific engineer raises the average further from 59.2% to 64.2% (+5.0 points); because these gains hold both before and after fine-tuning the target, Harness-R1 points toward co-evolving the harness engineer and the target agent.

Problem

• executable runtime harness
• agent failure trajectories
• harness editing
• lifecycle-wide adaptation
• reinforcement learning

Innovation

• runtime harness editing
• failure-conditioned learning
• online reinforcement learning
• co-evolution of agent and harness
• executable patch generation