DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees

2026-06-01

AI Summary

This work addresses the challenge of memory conflicts in large language model (LLM) agents during continuous interaction, where redundant experiences and minor environmental variations undermine decision consistency. To this end, the authors propose DeltaMem, a novel framework that introduces the concept of residual experience through a dual residual tree structure: one tree manages goal-oriented skills while the other organizes contextual knowledge, both sharing a common root of foundational experiences and incrementally encoding changes. DeltaMem integrates similarity-based retrieval with failure-aware penalties, root-to-match path reconstruction, and autonomous merging of frequently traversed paths to achieve experience deduplication, incremental updates, and automatic generalization. Experimental results demonstrate that DeltaMem significantly outperforms existing approaches across diverse interactive environments, effectively enhancing both memory efficiency and decision consistency.

Abstract

Large Language Model (LLM)-based agents increasingly rely on memory to learn from experiences over continual interactions. However, storing experiences as independent, flat units leads to substantial redundancy and retrieval conflicts, as similar episodes repeat overlapping content and subtle scene variations cause retrieved memories to offer contradictory guidance. To address this, we introduce residual experience, positing that newly acquired experience is often an incremental variation of existing knowledge. We propose DeltaMem, a framework that organizes experience memory into two independent residual trees, one storing goal-conditioned task experience as reusable skills and another for scene-level environment knowledge. Each tree uses a root node for generalized base experiences and incremental delta nodes for subsequent variations, allowing related experiences to share a common foundation without duplication. For retrieval, a failure-penalized similarity scan locates the best match, reconstructing the full experience via root-to-match chain composition. An autonomous consolidation mechanism distills high-frequency paths into new root nodes, enabling the trees to self-organize from general heuristics to specialized variants. Experiments across diverse interactive environments show that DeltaMem consistently outperforms existing baselines. To facilitate future research, we release the code at https://github.com/import-myself/DeltaMem.

Problem

• experience memory
• redundancy
• retrieval conflict
• continual learning
• LLM agents

Innovation

• residual experience
• memory organization
• incremental learning
• LLM agents
• self-organizing memory