DWM: Separating World Effects from Actions in Latent World Models

2026-07-21

AI Summary

Existing latent-variable world models struggle to disentangle action-driven changes from intrinsic environmental dynamics, resulting in confounded transfer signals and limited generalization. This work proposes Decoupled World Models (DWM), which, for the first time within a latent-variable framework, explicitly separate action-agnostic world effects from action-driven components through an action-invariance constraint and an orthogonal decomposition mechanism. DWM achieves this separation without modifying the backbone architecture by incorporating an auxiliary world prediction head, a normalized world contrastive loss, and orthogonality constraints. Evaluated on three new benchmarks—PushT-W, Reacher-W, and TwoRoom-W—DWM improves the success rate of CEM-based planning by 13.1% on average while maintaining original task performance.

Abstract

Latent world models underpin much of modern model-based control, yet current action-conditioned formulations supervise the next-latent transition with a single, undifferentiated target, forcing a monolithic learning signal to absorb every source of state change. In real world, however, transitions arise from two heterogeneous sources: an action-driven component induced by the agent, and an action-invariant world effect -- the change that would still occur under a null action, dictated by the environment's intrinsic dynamics (e.g., gravity-driven sliding, inertia, contact rebound, and persistent drift). Fusing them into a single target entangles the two inside the latent transition, prevents the model from attributing observed changes to their underlying causes, and undermines the transferability of the learned dynamics. We introduce DWM (Decomposed World Model), a supervision-level framework that operationalizes this decomposition. DWM augments the predictor of a latent world model with an auxiliary world head, regularized by a normalized world-contrastive objective to be action-invariant, while the original pred head is coupled to it via an orthogonality constraint; together, the two signals induce an explicit additive decomposition of the predicted transition into an action-invariant and a complementary action-driven component, without altering the underlying architecture or inference pipeline. To evaluate DWM under persistent world effects, we construct W-variants of three standard control benchmarks -- PushT-W, Reacher-W, and TwoRoom-W -- each instantiating a distinct action-invariant dynamic. DWM matches strong baselines on the flat counterparts and delivers a mean absolute improvement of 13.1% in CEM planning success across the W-variants.

Problem

• latent world models
• action-conditioned learning
• world effects
• state transition decomposition
• environment dynamics

Innovation

• Decomposed World Model
• action-invariant dynamics
• latent world models
• world-contrastive learning
• orthogonality constraint