FLEX: Continuous Agent Evolution via Forward Learning from Experience

2025-11-09

AI Summary

Current LLM-based autonomous agents are statically fixed after deployment, lacking biologically inspired continual evolution capabilities. This paper proposes the “forward experiential learning” paradigm, which constructs a structured experience repository enabling agents to continually learn through reflection on interaction outcomes—without gradient-based parameter updates. Methodologically, it integrates gradient-free learning, dynamic reflective reasoning, and structured experience modeling. Key contributions include: (1) demonstrating cross-agent transferability and inheritance of experiences; and (2) uncovering a scaling law governing experience growth, thereby supporting scalable, open-ended evolution. Evaluated on mathematical reasoning, chemical retrosynthetic planning, and protein fitness prediction, our approach achieves performance gains of 12–23%, validating both the effectiveness and generalizability of the continual evolution paradigm.

Technology Category

• Machine Learning: Evolutionary Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Autonomous agents driven by Large Language Models (LLMs) have revolutionized reasoning and problem-solving but remain static after training, unable to grow with experience as intelligent beings do during deployment. We introduce Forward Learning with EXperience (FLEX), a gradient-free learning paradigm that enables LLM agents to continuously evolve through accumulated experience. Specifically, FLEX cultivates scalable and inheritable evolution by constructing a structured experience library through continual reflection on successes and failures during interaction with the environment. FLEX delivers substantial improvements on mathematical reasoning, chemical retrosynthesis, and protein fitness prediction (up to 23% on AIME25, 10% on USPTO50k, and 14% on ProteinGym). We further identify a clear scaling law of experiential growth and the phenomenon of experience inheritance across agents, marking a step toward scalable and inheritable continuous agent evolution. Project Page: https://flex-gensi-thuair.github.io.

Problem

• Enabling continuous evolution of LLM agents through experiential learning
• Addressing static post-training limitations via gradient-free forward learning
• Developing scalable experience inheritance for autonomous agent growth

Innovation

• Continuous evolution via forward learning from experience
• Gradient-free learning paradigm for LLM agents
• Structured experience library through reflection on interactions