Hierarchical Graph Memory for LLM Agents with Path-level Localization and Rewrite

2026-08-05

AI Summary

This work addresses the limitations of existing graph-based memory approaches, which employ flat structures that introduce excessive irrelevant context during retrieval and struggle to efficiently capture dynamic relationships among memory units. To overcome these challenges, we propose HiGram, a hierarchical graph memory framework that organizes nodes from coarse to fine granularity, leverages MicroGraph-based path-level evidence localization, and incorporates a collaborative rewriting mechanism. This design enables precise memory updates and effective multi-hop reasoning. HiGram substantially improves memory relevance, update efficiency, and dependency consistency, outperforming current baselines in long-term conversational question answering and conflict-aware memory tasks across key metrics—including answer quality, evidence accuracy, and token efficiency.

Abstract

Agents for long term reasoning require a memory that can be efficiently and effectively updated over time, as new facts and external feedback continue to arrive. Recently, graph memory has been adopted to offer structural organization for multi-hop retrieval and reasoning. However, existing methods store all memories in a flat graph, and accumulated historical memories can introduce irrelevant contexts and increase the cost of evidence selection during retrieval. Moreover, they typically update memory units independently, requiring repeated unit-wise rewrite to cover related changes. To address these issues, we propose HiGram, an evolving hierarchical graph memory framework with path-level localization and rewriting. Specifically, we first propose a hierarchical graph memory, which organizes the memory into coarse-to-fine architecture composed of upper-level nodes and MemoryUnits, thereby reducing the amount of irrelevant information during retrieval. We further propose MicroGraph-based path-level localization, which leverages query and update conditioned MicroGraphs to identify support subgraph and evidence path before rewrite. Finally, we propose a coordinated rewriting method that jointly revises intra-unit memory and inter-unit dependencies, enable valid dependency structures updating in the localized evidence path. Experiments on benchmarks for long-term conversational question answering and conflict-aware memory evaluation demonstrate that our method demonstrate substantial improvements over baselines in answer quality and token efficiency. Besides, our method improves answer accuracy and query-valid evidence selection under dynamic, static, and conditional conflicts.

Problem

• graph memory
• long-term reasoning
• memory update
• evidence selection
• hierarchical structure

Innovation

• Hierarchical Graph Memory
• Path-level Localization
• Coordinated Rewriting
• Memory Units
• MicroGraph