CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

2026-07-15

AI Summary

This work addresses the limited generalizability of existing memory management approaches, which rely on question-answer pairs generated by large language models to assess memory value and are thus constrained by specific queries and downstream readers. To overcome this, the authors propose CMI-Mem, a lightweight, reinforcement learning–based memory management framework that introduces conditional mutual information (CMI) as a task-agnostic intrinsic reward signal. By combining CMI with downstream question-answering accuracy into a hybrid reward mechanism, CMI-Mem enables query-free evaluation of memory utility. This approach significantly enhances the generality, stability, and long-term effectiveness of memory management, consistently outperforming existing methods in memory retention and retrieval across diverse downstream tasks.

Abstract

Memory Manager models are pivotal in agent systems. Existing methods rely predominantly on LLM-judged synthetic question-answer (QA) pairs, making memory valuation dependent on sampled queries and the downstream reader. To address this limitation, we propose \textbf{CMI-Mem}, a reinforcement learning(RL)-based lightweight memory manager model with a hybrid reward that combines downstream QA correctness and intrinsic Conditional Mutual Information (CMI). CMI evaluates the information contributed by new conversational inputs relative to the current memory state without conditioning on a sampled QA query, thereby complementing rather than replacing QA grounding. Our codes are available at: https://github.com/Wyb0627/CMIMem , and the CMI-Mem-4B model checkpoint is available at: https://www.modelscope.cn/models/wyb0627/CMIMem-4B

Problem

• memory management
• generalization
• long-term memory
• reinforcement learning
• Conditional Mutual Information

Innovation

• Conditional Mutual Information
• Reinforcement Learning
• Memory Management
• Generalizable Memory
• Hybrid Reward