Real-Time Procedural Learning From Experience for AI Agents

2025-11-26

AI Summary

Existing LLM-based agents lack the ability to acquire procedural knowledge in real time through trial-and-error after deployment. This paper introduces PRAXIS, a lightweight framework for real-time experiential learning tailored to AI agents. PRAXIS enables online storage and retrieval of state-action-outcome exemplars by jointly matching environmental observations with internal agent states. It features a state-indexed episodic memory module and integrates retrieval-augmented generation to dynamically reuse past interactions for improved action selection. Crucially, PRAXIS requires no fine-tuning or external training, supporting plug-and-play post-deployment learning. Evaluated on the REAL web browsing benchmark, PRAXIS significantly improves task completion accuracy (+12.3%), execution stability, and cost efficiency. Moreover, it demonstrates cross-task generalization to unseen tasks—without any task-specific adaptation.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-training learning mechanism that stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state. PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time. When evaluated on the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments. These results demonstrate that PRAXIS enables the practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively.

Problem

• Enables AI agents to learn procedural knowledge from real-time trial and error
• Stores and retrieves action consequences by matching past and current states
• Improves task completion accuracy, reliability, and cost efficiency in web browsing

Innovation

• Lightweight post-training learning mechanism for agents
• Stores and retrieves action consequences by state matching
• Augments action selection with real-time exemplars