CAST: Reconstruction-Coupled Acceleration of Interactive World Models

2026-09-28

AI Summary

This study addresses the missing control responses and scene inconsistencies caused by frame skipping in accelerating interactive world models. To mitigate these issues, we propose a reconstruction-coupled inference framework that selects anchor points based on interaction sensitivity. Leveraging the phase-sensitive characteristics of low-frequency interpolation errors, the framework introduces a Phase-Aware Reconstruction (PAR) mechanism, which combines frequency and confidence signals to recover residuals while coordinating historical KV cache routing. This design effectively balances inference speed, generation quality, and interactive responsiveness. Evaluated on benchmarks such as Matrix-Game, the proposed method achieves 2.15× to 3.48× acceleration while preserving near-native visual fidelity, yielding state-of-the-art VBench scores.

Abstract

Interactive world models must respond quickly to controls while preserving scene consistency. Existing acceleration methods can miss heterogeneous control responses and spatial transport when recovering skipped features. We observe that interaction-induced feature changes correlate with approximation error, while low-frequency interpolation errors are phase-sensitive and show more predictable phase progression. These findings motivate CAST, a reconstruction-coupled inference framework. CAST selects anchors by interaction sensitivity and cross-layer coverage, reconstructs skipped residuals with frequency- and confidence-aware Phase-Aware Reconstruction (PAR), and coordinates historical KV routing according to downstream reconstruction responsibility. On Matrix-Game 3.0 and HY-World 1.5, CAST achieves 2.15x and 3.48x speedups, respectively, while maintaining visual quality close to Native (Figure 1). It also attains the highest VBench scores among compared methods and leads non-native baselines on seven and six of thirteen WorldMark dimensions, demonstrating a balance of generation speed, visual quality, and interactive responsiveness under real-time control. Code is available at https://github.com/lokiniuniu/CAST.

Problem

• Interactive World Models
• Inference Acceleration
• Scene Consistency
• Real-time Control
• Feature Reconstruction

Innovation

• Interactive World Models
• Reconstruction-Coupled Inference
• Phase-Aware Reconstruction
• Inference Acceleration
• KV Routing