Turbo Harness: Instance-Adaptive Harness Optimization

2026-09-30

AI Summary

This study addresses the limitation of existing globally unified testing frameworks, which achieve average optimality yet remain suboptimal for individual instances and struggle to adapt to specific task cases. To overcome this, we propose the first adaptive framework that recycles information from global optimization experiences. Methodologically, our approach repurposes global optimization artifacts to generate structured manuals and trains an editor to craft instance-aware patches for each case, thereby enabling dynamic framework generation and optimization. We evaluate the proposed method across seven benchmarks encompassing interactive agents, software engineering, and long-horizon terminal tasks. Experimental results demonstrate that our approach consistently outperforms existing baselines, establishing a robust solution for instance-level adaptation in complex testing environments.

Abstract

Automating the search for effective harnesses is an important step toward enabling agents to recursively self-improve. Existing harness optimizations typically produce a single global harness that is applied uniformly across task instances. However, a harness that works well on average may not be optimal for every instance. We introduce Turbo Harness, a framework that can adapt a globally optimized harness to each instance by reusing information generated during the original optimization process. Specifically, Turbo Harness recycles artifacts produced during a completed global harness optimization run, and summarizes them into a structured playbook. We train a harness editor to leverage this prior optimization experience to generate instance-specific patches to the global harness. At inference time, the editor uses the instance and the playbook to construct a tailored harness in which the execution model operates. Through numerical experiments, we show that Turbo Harness consistently outperforms existing harness optimization baselines across seven benchmarks spanning interactive agent tasks, software engineering, and long-horizon terminal tasks.

Problem

• harness optimization
• instance-adaptive
• AI agents
• self-improvement
• prompt engineering

Innovation

• Instance-Adaptive Optimization
• Harness Editor
• Structured Playbook
• Recursive Self-Improvement
• Patch Generation