continual learning

Existing LLM-driven automated visual modeling approaches rely on global, one-shot optimization, resulting in poor attribution, slow convergence, low stability, and limited accessibility for non-experts. Method: We propose an “iterative single-component fine-tuning” strategy, inspired by expert human practice, wherein only one module in the pipeline is optimized per iteration. This is integrated with training-feedback-guided modular updates, zero-shot prompt engineering, and a multi-domain evaluation protocol to construct an end-to-end LLM agent framework. Contribution/Results: Our approach significantly enhances interpretability, stability, and convergence efficiency of optimization. Evaluated across multiple standard benchmarks and Kaggle datasets, it consistently outperforms state-of-the-art zero-shot LLM methods, achieving superior classification accuracy and generalization capability.

Traditional continual learning relies on gradient-based parameter updates, making it ill-suited for real-time adaptation of deployed intelligent agents. This paper introduces ATLAS—a parameter-free, system-level orchestration framework for continual learning—that achieves dynamic self-adaptation without retraining. ATLAS decouples reasoning from execution, employs persistent memory modeling, and implements causal-aware runtime policy scheduling (e.g., adjusting supervision intensity and initial planning selection). Its core innovation shifts the continual learning paradigm from model parameter updating to structured, architecture-driven inference control, formalizing “adaptive efficiency” as a joint objective balancing task success rate and computational cost. Evaluated on ExCyTIn-Bench, ATLAS enables a frozen GPT-5-mini to achieve 54.1% task success—outperforming larger baseline models by 13%—while reducing computational cost by 86% and significantly enhancing cross-event generalization.

This work addresses the performance degradation of small language models in continual personalization caused by catastrophic forgetting. To mitigate this issue, the authors propose a continual learning framework based on sequential LoRA fine-tuning. The approach preserves checkpoints from each learning phase and employs retrospective evaluation across current tasks, historical tasks, and a fixed lightweight reference set. Central to the framework is a stability monitoring mechanism grounded in distributional diagnostics of the reference set, which effectively uncovers harmful adaptation behaviors that are not apparent through task-specific metrics alone. This mechanism successfully identifies unstable patterns emerging during sequential personalization, offering a novel perspective for evaluating and monitoring stability in continual learning scenarios.

To address catastrophic forgetting and inefficient knowledge utilization in continual reinforcement learning (CRL) under non-stationary environments, this paper proposes the Continual Knowledge Adaptation framework for RL (CKA-RL). Methodologically, CKA-RL constructs a task-specific knowledge vector pool and employs gradient-based analysis to identify critical parameters, enabling parameter-level knowledge preservation and selective transfer. It further introduces a dynamic knowledge matching and adaptive fusion mechanism that balances storage efficiency with retention of essential information. The framework supports efficient accumulation, reuse, and cross-task transfer of historical knowledge. Extensive experiments on three standard benchmarks demonstrate that CKA-RL significantly outperforms existing state-of-the-art methods: it achieves a 4.20% improvement in overall performance and an 8.02% gain in forward transfer, thereby enhancing scalability and generalization capability in continual RL.

This study addresses the dual challenges of emerging domains and data drift faced by large language models in dynamic environments. The authors decompose continual learning into spatial (new domains) and temporal (data drift) dimensions and propose the first mechanism-agnostic, unified evaluation protocol. Within this consistent framework, they systematically compare the adaptability of diverse approaches—including prompt engineering (e.g., GEPA, ACE), supervised fine-tuning (SFT, SDFT), online reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Their analysis reveals that effective adaptation depends on aligning update mechanisms with specific environmental dynamics: online reinforcement learning excels at knowledge updating yet is sensitive to noise; distillation-based methods offer stability but struggle to correct outdated facts; and prompt-based strategies respond rapidly but exhibit limited generalization.

Current foundation models exhibit significant limitations in continual learning, experience accumulation, and personalization, with conventional weight-update-based approaches particularly prone to catastrophic forgetting. This work proposes a novel architecture centered on modular memory that systematically integrates in-weight learning (IWL) and in-context learning (ICL) for the first time: ICL enables rapid acquisition of new knowledge, while IWL provides stable, long-term capability enhancement, and modular memory serves as a synergistic bridge between the two. The resulting framework offers an innovative and practical pathway toward building agents capable of lifelong evolution, continuous adaptation, and personalized interaction.

This work addresses the limitation of current large language models in competitive programming, which typically rely on single-pass generation and thus fail to leverage their iterative refinement potential. The authors propose a self-refinement approach that integrates a Skeptical-Agent mechanism with a lightweight reinforcement learning framework. The Skeptical Agent critically filters generated code through local execution-based verification, while the reinforcement learning component requires only standard RLVR data for training. By enabling multiple refinement iterations beyond the single-attempt paradigm, the method substantially enhances model performance: a fine-tuned 4B-parameter model surpasses the single-pass results of a 32B model and approaches those of a 235B model.

This work addresses the limited metacognitive awareness of task progress in existing large language model (LLM) agents within reinforcement learning settings, which hinders their performance on long-horizon tasks. To overcome this limitation, the authors propose the RePro framework, which introduces a novel retrospective progress-aware mechanism: after forward execution, the agent retrospectively analyzes its trajectory, self-evaluates the contribution of each step toward task completion, and generates intrinsic progress signals. Coupled with Retrospection Warmup for initialization and the RePro-PO policy optimization algorithm, the approach enables effective training with minimal external demonstrations. Experimental results demonstrate that RePro significantly enhances the performance of Qwen-series models across WebShop, ALFWorld, and Sokoban benchmarks, yielding up to a 12% absolute improvement in task success rate.

This study addresses a critical challenge in the continuous self-evolution of large language model agents: the tendency to lose previously acquired capabilities while adapting to new tasks. The authors formally identify and term this phenomenon “capability erosion in self-evolution.” To mitigate this issue across four key dimensions—workflow, skills, model parameters, and memory—they propose a general Capability-Preserving Evolution (CPE) principle that explicitly constrains destructive capability drift, thereby enabling the acquisition of new competencies without compromising prior performance. Experimental results demonstrate that CPE significantly enhances stability in models such as GPT-5.1, increasing the retention rate of simple tasks in workflow evolution from 41.8% to 52.8% while simultaneously improving adaptability to complex tasks.

Existing static human-curated datasets are difficult to scale, while raw interaction logs suffer from high noise and sparse information, limiting model training efficacy. This work proposes Echo, a novel framework that, for the first time, models user revision sequences of AI-generated outputs as a knowledge distillation process, establishing a general paradigm for extracting high-quality training signals from real-world interaction data. By integrating refined sequence extraction, continual learning, and log alignment techniques, Echo effectively overcomes the bottlenecks imposed by static datasets. Evaluated in a production code-completion environment, the approach significantly improves model acceptance rates from 25.7% to 35.7%, demonstrating its capability for continuous improvement and practical effectiveness.