HLA-WM: Hybrid Linear Attention for Long-Horizon Video World Models

2026-10-05

AI Summary

This study addresses the long-range scene forgetting problem caused by linear attention in long-horizon video world models by proposing a training-free hybrid memory framework. Methodologically, it pioneers the use of the Gated Delta Network (GDN) affine structure to cache block-wise summaries, integrating camera geometry-guided coarse-grained retrieval with fine-grained recurrent state reconstruction to achieve a selective, addressable memory mechanism. Experimental results demonstrate that this approach significantly improves PSNR and reduces rotation error while decreasing memory consumption by 12× with less than 1.6% throughput overhead. Furthermore, the proposed method exhibits superior cross-scene generalization capabilities.

Abstract

Long-horizon video world models require persistent memory to preserve scene consistency over extended rollouts. Softmax attention retains the full generation history through a growing KV cache, whereas recurrent linear attention compresses history into fixed-size states with substantially lower memory cost. However, we identify severe long-range forgetting in Gated DeltaNet (GDN), where information from distant but relevant scenes is progressively attenuated by subsequent state updates. To address this limitation, we propose HLA-WM, a training-free hybrid linear-attention framework that combines coarse-grained geometry-guided retrieval with fine-grained recurrent linear-state computation. HLA-WM exploits the affine structure of GDN to cache compact chunk-wise transition summaries, retrieve scene-relevant historical chunks using camera geometry, and recompose them into query-specific recurrent states. On the $60$-second SANA-WM-Bench, HLA-WM improves all six aggregate revisit-consistency and camera-control metrics of the base autoregressive generator without additional training, including a $0.74$ dB PSNR gain and a $28.5\%$ reduction in rotation error. The improvements persist after downstream refinement and generalize to MBench-A, where HLA-WM consistently improves all three revisit-consistency metrics across all four subsets and all evaluated inference modes over $547$ samples. At a $60$-second context, HLA-WM reduces historical-state memory by $12\times$ relative to full KV caching while incurring at most a $1.6\%$ reduction in inference throughput. These results demonstrate that selectively addressable recurrent memory can improve long-range scene recall while preserving the efficiency advantages of GDN. Project page: https://caesarhhh.github.io/hla-wm/

Problem

• long-horizon video world models
• long-range forgetting
• scene consistency
• recurrent linear attention
• Gated DeltaNet

Innovation

• Hybrid Linear Attention
• Video World Models
• Gated DeltaNet
• Geometry-guided Retrieval
• Training-free