Constructing coherent spatial memory in LLM agents through graph rectification

2025-10-05

AI Summary

This work addresses the challenge of constructing coherent spatial memory for LLM-based agents in large-scale environments, specifically focusing on detecting, localizing, and rectifying structural inconsistencies during incremental navigation graph construction. We propose a dynamic map repair framework integrating version control, topological consistency verification, and edge impact scoring: (1) versioned graph editing history ensures full traceability; (2) an edge impact scoring mechanism identifies critical conflicting edges, enabling prioritized correction of high-impact errors; and (3) fine-grained rollback and quantitative repair evaluation are supported. The framework enables efficient and robust consistency maintenance in context-driven incremental mapping, significantly improving map correctness—especially under complex entanglement and cascading inconsistency scenarios. Extensive evaluation on an enhanced MANGO benchmark demonstrates both effectiveness and generalizability across diverse mapping conditions.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: Localization, Mapping, and Navigation
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Given a map description through global traversal navigation instructions (e.g., visiting each room sequentially with action signals such as north, west, etc.), an LLM can often infer the implicit spatial layout of the environment and answer user queries by providing a shortest path from a start to a destination (for instance, navigating from the lobby to a meeting room via the hall and elevator). However, such context-dependent querying becomes incapable as the environment grows much longer, motivating the need for incremental map construction that builds a complete topological graph from stepwise observations. We propose a framework for LLM-driven construction and map repair, designed to detect, localize, and correct structural inconsistencies in incrementally constructed navigation graphs. Central to our method is the Version Control, which records the full history of graph edits and their source observations, enabling fine-grained rollback, conflict tracing, and repair evaluation. We further introduce an Edge Impact Score to prioritize minimal-cost repairs based on structural reachability, path usage, and conflict propagation. To properly evaluate our approach, we create a refined version of the MANGO benchmark dataset by systematically removing non-topological actions and inherent structural conflicts, providing a cleaner testbed for LLM-driven construction and map repair. Our approach significantly improves map correctness and robustness, especially in scenarios with entangled or chained inconsistencies. Our results highlight the importance of introspective, history-aware repair mechanisms for maintaining coherent spatial memory in LLM agents.

Problem

• Detecting and correcting structural inconsistencies in LLM-built navigation graphs
• Building complete topological maps from incremental stepwise observations
• Maintaining coherent spatial memory through history-aware graph repair mechanisms

Innovation

• Uses Version Control for graph edit history tracking
• Introduces Edge Impact Score for repair prioritization
• Refines benchmark dataset by removing non-topological conflicts