DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds

2026-08-06

AI Summary

This work addresses the limited cross-scaffold generalization of current CLI agents, which perform well on specific training scaffolds like OpenHands but degrade significantly on others due to over-reliance on explicit planning structures during fine-tuning. To overcome this, the authors propose DCAS, a method that decouples the scaffold from the backend model via an API interception layer that requires no scaffold modification, thereby internalizing both explicit and implicit planning capabilities. For the first time, the study distinguishes between these two planning mechanisms and integrates planning-aware trajectory collection with fine-tuning, achieving substantial improvements in cross-scaffold generalization even with limited data. Experiments demonstrate that models fine-tuned with DCAS consistently outperform baselines on unseen scaffolds, highlighting the critical role of planning quality in cross-environment transfer.

Abstract

CLI-based software-engineering agents have matured rapidly, yet the open ecosystem has converged on a single training environment: trajectory datasets used to fine-tune open models are collected almost exclusively under OpenHands. Models fine-tuned on this data score well under OpenHands but degrade substantially when deployed under any non-training scaffold. Untrained base models do not show this divergence, indicating the gap is fine-tuning-induced and tied to the conventions of the training scaffold. We argue that a load-bearing scaffold-specific behavior is planning structure, in two senses this paper distinguishes: explicit planning, a pre-execution plan produced as a first-class artifact, and implicit planning, the structural conventions that shape execution throughout the agent loop. Under this hypothesis, closing the gap requires moving planning from a fixed scaffold artifact to a learned model capability. We introduce Decoupling CLI Agent Scaffolding (DCAS), a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model without modifying the scaffold, enabling cross-scaffold evaluation and planning-aware trajectory collection. Using DCAS, a controlled plan-source intervention confirms planning quality is a high-leverage component, with gains exceeding the cross-scaffold drops we observe. A model fine-tuned on a small set of DCAS-collected planning-aware trajectories under a single scaffold gains consistently across non-training scaffolds, and the two senses of planning are empirically separable in training data.

Problem

• CLI agent
• scaffold generalization
• planning structure
• fine-tuning bias
• cross-scaffold evaluation

Innovation

• DCAS
• CLI agent
• planning decoupling
• scaffold generalization
• trajectory collection