Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory

2026-10-01

AI Summary

This study addresses the challenge of managing long-range spatial context caused by growing memory sequences in long-video world models. To this end, it proposes SMI, the first framework that leverages understanding models to systematically manage spatial memory. Specifically, SMI pioneers the integration of the spatial reasoning capabilities of multimodal large language models into world model memory management. It achieves efficient memory optimization through four synergistic atomic operations: spatial clustering, intra-cluster sparsification, action-aware retrieval, and reliability filtering. Experimental results demonstrate that SMI significantly improves memory sparsity, generation stability, and spatial consistency across multiple benchmarks and backbone architectures.

Abstract

Long-video generation and world models have shown strong potential for interactive entertainment and embodied simulation by predicting future observations conditioned on user actions and historical memory. However, as memory sequences grow longer and their structures become increasingly complex, managing long-range spatial context becomes increasingly challenging, calling for a more intelligent and systematic memory-management strategy. Building on the advancing spatial reasoning capabilities of multimodal large language models (MLLMs) and the broader vision of unified models, we propose Spatial Memory Intelligence (SMI), the first framework to systematically employ an understanding model for spatial-memory management in long-video world models. SMI introduces four coordinated atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. Extensive experiments across multiple baselines, benchmarks, and world-model backbones demonstrate the effectiveness and generalizability of SMI, achieving comprehensive improvements in memory sparsity, generation stability, and spatial consistency.

Problem

• world models
• long-video generation
• spatial memory management
• long-range spatial context

Innovation

• Spatial Memory Intelligence
• World Models
• Long-video Generation
• Multimodal Large Language Models
• Memory Management