CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG

2026-07-29

AI Summary

This work addresses the limitations of existing retrieval-augmented generation (RAG) systems in multi-turn information-seeking dialogues, where modeling cross-turn multi-hop reasoning and long-range dependencies remains challenging due to unstructured dialogue memory. To overcome this, the authors propose CMT-RAG, a novel framework that aligns dialogue memory with retrieval by dynamically constructing sub-question-level structured reasoning trajectories via a state-space trajectory generator. These trajectories are persistently stored as conversation-level directed acyclic graphs (DAGs), enabling efficient reuse in subsequent turns. The study also introduces MuMu-QA, a new benchmark dataset featuring explicit annotations of sub-question dependencies. Experimental results demonstrate that CMT-RAG significantly outperforms five representative baseline approaches on both MuMu-QA and established RAG benchmarks, confirming its effectiveness and superiority in multi-turn, multi-hop question answering.

Abstract

Multi-turn information-seeking conversations require both multi-hop reasoning and long-range dependency tracking across turns. However, existing RAG systems typically represent conversational memory as raw dialogue history, rewritten queries, or unstructured summaries, making it difficult to recover the specific prior reasoning steps and evidence required for follow-up queries. Our key insight is to align conversational memory with retrieval by representing dialogue context as sub-question-level reasoning traces. Building on this insight, we introduce MuMu-QA, a benchmark for multi-turn multi-hop RAG with explicit cross-turn sub-question dependency annotations, and CMT-RAG, a complementary memory framework for this setting. At each turn, CMT-RAG employs a state-space trace generator, whose recurrent state serves as runtime memory, to incorporate recent conversational context and decompose the current query into structured trace drafts containing retrieval-oriented sub-questions and dependencies on earlier traces. It then grounds these drafts with retrieved evidence and stores them as persistent memory traces in a session-level DAG, enabling future turns to efficiently recover relevant prior reasoning and evidence. Experiments on MuMu-QA and corpus-level RAG benchmarks show that CMT-RAG consistently outperforms five categories of RAG baselines in answer accuracy.

Problem

• multi-turn conversation
• multi-hop reasoning
• conversational memory
• retrieval-augmented generation
• long-range dependency

Innovation

• Complementary Memory Traces
• Multi-turn Multi-hop RAG
• Reasoning Traces
• State-Space Trace Generator
• Session-level DAG