AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents

2026-07-18

AI Summary

This work addresses the challenge of efficiently transferring knowledge from a powerful teacher model to a lightweight student agent under sparse binary feedback and weak actuator constraints. It proposes a novel knowledge distillation framework that operates without weight updates, expert demonstrations, or teacher involvement during inference. The approach introduces a failure-triggered Ralph Loop mechanism to generate environment-validated executable notes, a student-aware knowledge synthesis strategy that adapts teacher experiences to the student’s action granularity, and unsupervised interactive learning via external memory injection. Evaluated on code generation, mathematical reasoning, and tool-use tasks, the method significantly enhances student performance, with ablation studies confirming the contribution of each component, thereby establishing a new paradigm for deploying high-capability, lightweight LLM agents.

Abstract

Deploying LLM agents typically requires a compact test-time student, even if a stronger teacher is available during training. We study knowledge brewing: distilling a teacher's interactive experience into a persistent external memory for the student. Crucially, this requires no weight updates, expert demonstrations, ground-truth labels, or test-time teacher access. This setting poses two challenges: environments provide only sparse, binary feedback, and teacher-authored notes must be inherently tailored to be concretely executable by a substantially weaker student. To address these hurdles, we propose AgentBrew, comprising two coupled components. First, a failure-triggered teacher--Ralph Loop mitigates sparse feedback by transforming student failures into environment-validated notes. Second, student-aware synthesis calibrates teacher knowledge to the weak executor's operational granularity, yielding model-specific, actionable guidance. Extensive evaluations and comprehensive ablations across coding, math, and tool-use tasks demonstrate that this asymmetric, training-free brewing paradigm produces highly capable yet deployable LLM agents.

Problem

• knowledge distillation
• LLM agents
• lifelong learning
• external memory
• student-teacher framework

Innovation

• knowledge distillation
• lifelong learning
• LLM agents
• external memory
• student-teacher framework