Harness Learning Enables Generalizable Test-Time Adaptation

2026-09-28

AI Summary

This study addresses the difficulty language model agents face in efficiently adapting execution frameworks to diverse tasks at test time. To this end, this work proposes "framework learning," which formulates framework revision as meta-learning over executable programs. Specifically, a proposer model is trained via reinforcement learning to iteratively refine a solver's code framework using execution feedback, thereby enabling test-time adaptation without parameter updates. By integrating large language model agents with program synthesis and automated repair techniques, this approach endows agents with the capacity to continuously generalize and improve from experience. Experimental results demonstrate significant performance gains on reasoning and multi-hop question answering tasks, validating that such test-time adaptation capabilities transfer effectively to unseen tasks.

Abstract

A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer with reinforcement learning, using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, without performing any parameter-space update. Experiments on reasoning and multi-hop question answering show that harness learning improves revision quality and that the ability to adapt at test time transfers to unseen tasks. Policies trained on individual revisions can continue improving harnesses over multiple rounds, while the benefits of training on revision sequences vary across settings. These findings suggest a path towards continually learning agents that turn accumulated experience into generalizable improvements.

Innovation

• Harness Learning
• Test-Time Adaptation
• Meta-Learning
• Reinforcement Learning
• Executable Programs