Harness-Aware Distillation for Small Language Model Agents

2026-10-02

AI Summary

This study addresses the limitation of existing agent distillation methods that overlook the baseline capabilities of the execution harness, which hinders smaller models from acquiring decision logic beyond such foundational capacities. To overcome this, we propose a harness-aware distillation framework that targets the teacher model’s incremental capabilities. By introducing an action-preference contrastive mechanism alongside validity verification, and leveraging online policy distillation to construct preference pairs while filtering contradictory samples, our approach achieves unsupervised incremental knowledge transfer without requiring task-specific rewards or labels. Extensive evaluations across multiple benchmarks demonstrate that the proposed method significantly outperforms baselines by effectively reducing unproductive loops and improving error recovery rates. These results validate its capacity to adaptively internalize learnable feedback directly into model weights.

Abstract

Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.

Problem

• Knowledge Distillation
• Language Model Agents
• Small Language Models
• Agent Harness

Innovation

• Harness-Aware Distillation
• Small Language Model Agents
• Action Preference
• Validity Check
• On-policy Distillation