Beyond Memory Construction: Rethinking Memory Access for LLM-based Conversational Agents

2026-09-27

AI Summary

This study addresses the limitations of conventional memory construction for large language model agents in long-horizon, high-entropy dialogues, including information loss, state instability, and prohibitive computational overhead. To this end, we propose Threader, a system that facilitates a paradigm shift from memory rewriting to structure-aware access. Threader preserves raw interactions as first-order memory while organizing context through lightweight incremental topic segmentation and multi-view representation learning. Furthermore, it integrates segmented retrieval with local evidence matching to ensure information integrity. Experimental results demonstrate that Threader significantly improves answer accuracy and evidence recall while substantially reducing the computational costs associated with memory construction.

Abstract

Memory is a core component of conversational agents, enabling coherent and context-aware behavior over long interactions. Recent approaches commonly rely on LLM-based memory construction, where raw interactions are rewritten into structured memory units and later retrieved via a RAG pipeline. While effective in controlled settings, we show that this paradigm breaks down in long-horizon, high-entropy conversations: memory construction becomes increasingly lossy and unstable as context length and information complexity grow, and incurs prohibitive cost due to repeated LLM invocation. To address these limitations, we propose Threader, a memory system that shifts the focus from memory construction to efficient, structure-aware access over raw interactions. Instead of rewriting interactions, Threader preserves them as first-class memory, organizes them into topic-coherent segments via lightweight incremental segmentation, and enables accurate retrieval through multi-view representation. At query time, it performs multi-signal retrieval that combines segment-level access with localized evidence matching, ensuring both completeness and coherence. Extensive experiments demonstrate that Threader consistently improves answer accuracy and evidence recall, while significantly reducing the memory construction overhead.

Problem

• conversational agents
• memory construction
• long-horizon conversations
• retrieval-augmented generation
• LLM overhead

Innovation

• Memory Access
• Incremental Segmentation
• Multi-view Representation
• Multi-signal Retrieval
• Conversational Agents