ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games

2026-07-29

AI Summary

Existing latent world models often suffer from “context collapse” in long-horizon planning, where future states generated by distinct action sequences become indistinguishable, thereby degrading planning performance. This work proposes ActSWM, which uniquely enforces action sensitivity as a core constraint—rather than an auxiliary objective—in latent rollouts. By adhering to a transition disentanglement principle, ActSWM constructs a dynamics model that preserves distinguishable latent trajectories for different actions and enables recovery of action information from local transitions. Integrated with an autoregressive architecture and model-predictive control (MPC), ActSWM achieves substantially higher success rates on long-horizon Minecraft tasks, maintains greater trajectory divergence conditioned on actions, and effectively reconstructs action sequences from offline videos.

Abstract

Latent world models support efficient model-predictive control by optimizing future control sequences in latent space and replanning in a receding-horizon manner. However, existing latent predictors often lack stable long-horizon rollout ability, and prediction accuracy alone does not ensure that rollouts remain responsive to the actions being planned. We identify Context Collapse, a failure mode in which autoregressive latent predictors maintain high similarity to future states while producing nearly indistinguishable futures under different action sequences. To address this issue, we propose ActSWM, an action-sensitive latent world model grounded in a transition-separation principle: a planning-useful latent dynamics model should keep alternative-action futures distinguishable and make the action associated with each local transition recoverable. Under this principle, action sensitivity is enforced as a constraint on latent rollouts rather than treated only as an auxiliary prediction target, encouraging predicted futures to preserve action-dependent differences over long horizons. Across step-drift analysis, closed-loop Minecraft planning, and cross-game local action recovery, ActSWM preserves larger action-dependent rollout gaps than existing baselines, improves task success in long-horizon interactive settings, and enables world-model-based action recovery from offline gameplay videos.

Problem

• Context Collapse
• latent world models
• action sensitivity
• long-horizon planning
• open-world games

Innovation

• Action-Sensitive World Models
• Context Collapse
• Transition-Separation Principle
• Latent Dynamics
• Long-Horizon Planning