MemToolAgent overview with a simple restaurant booking scenario where the agent retrieves similar memories, receives feedback on an invalid time format, and generates a reflection to update its memory

2026-06-05

AI Summary

This work addresses the challenge that large language model agents struggle to learn user preferences from historical interactions in tool-use tasks. To overcome this limitation without requiring fine-tuning, the authors propose a dynamic memory mechanism that enables agents to effectively leverage past experiences to refine their tool-calling behavior. The approach integrates a unified structured memory format, reflection-based memory extraction, and an adaptive retrieval strategy. Its core components include a structured memory store, a reflection generation module, and a dynamic retrieval module. Experimental results demonstrate significant improvements, with relative performance gains of 29%, 80%, and 17% on the WorkBench, NESTFUL, and PEToolBench benchmarks, respectively, substantially enhancing the agent’s capacity for personalized adaptation.

Abstract

Modern large language model (LLM) agents can use external tools to help users solve complex tasks. However, for problems that require learning from long-term historical events or from previous agent-environment interactions, LLM agents are required to use memory mechanisms to store and retrieve experiences. While sophisticated memory systems exist for dialogue agents, few studies have empirically examined how to improve agents' tool-using capabilities through past user-agent conversations. We propose MemToolAgent, a framework that improves tool use through memory management. Our approach contains a memory extraction module that processes past experiences into structured memory entries, and a retrieval module that dynamically selects a subset of the stored memory entries. This enables more personalized and accurate responses aligned with user preferences and feedback without requiring LLM fine-tuning. In summary, this work has three main contributions: (1) a unified memory entry format that improves both general-purpose and personalized tool use without LLM fine-tuning, (2) a reflection-based memory extraction that uses environment and user feedback to distill wrong executions into critiques to store, and (3) a retrieval module that chooses how many past experiences to use based on the memory similarity distribution. MemToolAgent achieves 29%, 80%, and 17% relative improvements compared to strong baselines on the WorkBench, NESTFUL, and PEToolBench benchmarks, respectively.

Problem

• tool use
• memory mechanism
• LLM agent
• user feedback
• experience learning

Innovation

• memory management
• tool use
• reflection-based learning
• LLM agents
• experience retrieval