Addressable Memory for Video World Models

2026-08-07

AI Summary

This work addresses the challenge of unreliable visual memory in interactive video world models during long-horizon generation, where positional encoding extrapolation failures and cache compression undermine visual persistence. The authors propose WorldTrace, a framework that—without requiring retraining—preserves cache addressability by assigning in-distribution virtual positions to compressed memory representations. WorldTrace supports two memory compression strategies: temporal coherence via WorldTrace-Field and event recall via WorldTrace-Landmark. This approach introduces the first training-free, addressable visual memory mechanism, effectively mitigating the phase conflict between RoPE extrapolation and compression. Evaluated on the newly introduced LoopBench benchmark, WorldTrace-Field improves temporal consistency by 15.5%, while WorldTrace-Landmark enhances event recall accuracy by 19.5%, significantly advancing long-range visually persistent generation.

Abstract

We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward previously generated frames. However, we find that models can no longer reliably address stored content once rollouts extend beyond the training horizon, because temporal Rotary Positional Embeddings (RoPE) offsets then fall outside the range seen during training and the model struggles to retrieve the relevant visual information through attention. Moreover, naively compressing the cache in the RoPE-rotated space corrupts memory by averaging together incompatible positional phases. To address this, we propose WorldTrace, a training-free memory framework for long-horizon visual persistence. WorldTrace keeps compressed memory addressable by assigning each summary slot a distinct, in-distribution virtual position. Within this addressable cache, we study two memory compression approaches: WorldTrace-Field compresses history for temporal coherence, while WorldTrace-Landmark stores verbatim scene traces at detected transitions for episodic recall. We further introduce LoopBench, a benchmark evaluating whether a compressed cache can reconstruct a previously visited scene after a long detour. WorldTrace-Field improves temporal consistency by +15.5%, and WorldTrace-Landmark improves episodic recall by +19.5% on LoopBench, extending visually persistent generation without retraining.

Problem

• visual persistence
• video world models
• KV cache
• RoPE
• memory compression

Innovation

• addressable memory
• video world models
• memory compression
• Rotary Positional Embeddings
• visual persistence