StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation

2026-07-29

AI Summary

This work addresses a critical limitation in existing game world models, which, despite generating visually realistic environments, often neglect internal game states—such as health points or skill cooldowns—leading to rule violations. To resolve this, we propose StatePlay, the first game world model that explicitly represents internal states and jointly predicts both visual frames and game state variables. By conditioning image generation on predicted states, StatePlay ensures consistency with underlying game mechanics. We introduce a hybrid Transformer architecture that separately optimizes visual and state prediction branches while training them end-to-end via a multimodal objective function. Experimental results demonstrate that StatePlay achieves an average normalized L1 distance below 0.06 for state prediction and improves mechanistic fidelity of generated sequences by 18.6% compared to state-agnostic baselines.

Abstract

Recent game world models can generate visually realistic and interactive environments conditioned on player actions. However, games are not defined by pixels alone; they are governed by explicit mechanics, namely state-dependent rules that control health reduction, skill activation, and game termination. These mechanics depend on precise internal states, such as health points, skill meters, and timers, which are tightly coupled with visual observations and determine how gameplay evolves. Without modeling these state dynamics, existing game world models may generate visually plausible rollouts but violate the underlying game rules. In this paper, we propose StatePlay, a novel state-aware game world model that jointly predicts visual content and game states to promote mechanics-consistent generation. StatePlay adopts a mixture-of-transformers (MoT)-style architecture that preserves specialized visual and state representations while enabling cross-modal interaction, allowing predicted states to guide frame generation. Each branch is further optimized with a distinct objective suited to its modality. Experiments show that StatePlay achieves an average normalized L1 distance below 0.06 for state prediction. Furthermore, compared with models without explicit state modeling, our method improves mechanics fidelity in generated game rollouts by 18.6%. Overall, our work highlights the importance of state-aware game world modeling and advances beyond pixel-level realism toward complete and mechanically faithful game generation.

Problem

• game world models
• game mechanics
• state dynamics
• mechanics-consistent generation
• internal states

Innovation

• state-aware modeling
• game world models
• mechanics-consistent generation
• mixture-of-transformers
• internal game states