Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents

2026-07-28

AI Summary

This work addresses a critical gap in existing approaches that typically treat tool selection or action sequences as control variables while overlooking explicit regulation of prompt context construction. For the first time, we formalize context assembly—encompassing prompt templates, example selection, and the amount of retrieved content—as controllable variables and introduce a novel dual-layer policy architecture. The outer layer employs either a contextual multi-armed bandit or REINFORCE algorithm for online context modulation, while the inner layer leverages a frozen large language model to execute downstream tasks. We establish a theoretical framework analyzing stability and uncertainty, proving that expected reward is non-decreasing under bounded policy updates. Empirical results further demonstrate that the controller’s confidence is well-calibrated with task performance, validating the efficacy of our approach.

Abstract

A growing body of 2026 work applies control theory to LLM agents: Lyapunov-certified stability for tool-mediated controllers (Prinos et al., "Stable Agentic Control", 2026), sample-complexity bounds for sparse policies over massive discrete tool universes (Majumdar, "Sparse Agentic Control", 2026), and regulatory-control decompositions of multi-agent systems into auditable feedback loops (Nogueira and Skogestad, 2026). We do not claim to introduce control theory to LLM agents -- that ship has sailed. Our narrower claim is about what the controlled variable is. Prior work controls tool selection, inter-agent message routing, or the agent's raw action stream. We instead treat context assembly itself -- which prompt template, which few-shot demonstrations, how much retrieved context, how many planning/verification passes -- as the controlled variable, learned online by a contextual bandit or REINFORCE policy sitting outside a frozen model. This paper develops the formal decomposition (inner frozen policy $π_θ$, outer context policy $π_φ$), gives a stability argument for the online controller in the sense used by Zhang et al. (2026) (non-decreasing expected reward under bounded policy change), and reports an uncertainty-calibration analysis of the controller's own confidence against realized task outcomes. The applied counterpart to this paper instantiates the same controller across three domains and two model providers and releases the dataset, trajectory logs, and a deployment recipe; here we focus on the formal framing and the stability/uncertainty evidence a control-theoretic claim requires.

Problem

• context assembly
• controlled variable
• frozen LLM agents
• control theory
• prompt engineering

Innovation

• context assembly
• frozen LLM agents
• control-theoretic framing
• outer context policy
• online contextual control