world model training

Generative models suffer from catastrophic forgetting in continual learning. This paper systematically surveys recent advances in continual learning for large language models, multimodal models, vision-language-action models, and diffusion models. We propose the first brain-inspired memory mechanism-based taxonomy—comprising architectural, regularization-based, and replay-based methods—and establish their unified classification. We further develop a comprehensive analytical framework for generative models, covering training objectives, benchmarks, and backbone architectures, integrating techniques such as knowledge distillation, elastic weight consolidation, experience replay, parameter isolation, and gradient projection. A task- and domain-incremental evaluation protocol is designed to support rigorous assessment. The survey encompasses over 100 state-of-the-art works and introduces *Awesome-CL-in-GMs*, the first open-source repository dedicated to continual learning in generative models—providing both theoretical foundations and practical guidelines for algorithmic innovation and real-world deployment.

This work addresses the limitations of current world models, which often prioritize visual fidelity at the expense of physical plausibility and causal structure, thereby hindering their capacity for intervention, long-horizon prediction, and safety-critical decision-making. To overcome these shortcomings, the paper proposes a novel paradigm grounded in physical realism and explicit causal modeling, reframing world models as actionable simulators. The approach integrates a structured 4D interface, constraint-aware dynamics, and counterfactual reasoning mechanisms to enable precise intervention planning. Furthermore, it introduces a closed-loop evaluation framework to rigorously assess model performance. Evaluated in high-stakes domains such as medical decision-making, the method demonstrates substantial improvements in long-term robustness, intervention efficacy, and causal consistency.

This work addresses the quadratic computational and memory overhead of conventional Transformers with respect to context length, which hinders efficient predictive understanding and controllable simulation in time-series modeling. To overcome this limitation, the authors propose World Machine—a generative world model grounded in a structured latent state mechanism that circumvents the quadratic complexity of standard Transformers and dynamically adapts to both observation volume and context length. The approach integrates a modified Transformer architecture, latent state representation learning, and a tailored training protocol. Evaluated on a custom Toy1D synthetic dataset, World Machine demonstrates superior modeling capacity compared to baseline Transformers, with ablation studies confirming the contribution of each component to overall performance gains.

Existing pre-trained video generation models rely on static prompts (e.g., text or images), limiting their ability to model interactive, dynamic scenes. To address this, we propose the Dynamic World Simulation (DWS) framework, which transforms video generation models into interactive world simulators. DWS introduces a lightweight, universal action-conditioning module that drives scene evolution according to given action trajectories; a motion-augmented loss that explicitly optimizes dynamic consistency—rather than pixel-level fidelity; and a priority imagination sampling strategy to enhance long-horizon temporal controllability. The framework is architecture-agnostic, supporting both diffusion models and autoregressive Transformers. Experiments demonstrate that DWS significantly improves action controllability and dynamic coherence in game and robotics simulation scenarios. Moreover, when applied to downstream model-predictive control tasks, DWS achieves state-of-the-art sample efficiency.

This work addresses the “world stability” deficiency in diffusion-based world models—specifically, their inability to maintain cross-step scene consistency during temporal generation, which hinders deployment in reinforcement learning and safety-critical applications. We first formally define “world stability” and propose an inverse-action backtracking evaluation framework. This framework systematically quantifies stability via temporal observation comparison, revealing significant degradation in state-of-the-art diffusion world models. We further empirically validate multiple stabilization strategies; the best-performing method improves backtracking consistency by 37%. Our contributions include: (i) the first formal definition of world stability for diffusion-based world models; (ii) a principled, observation-driven evaluation framework enabling rigorous stability assessment; and (iii) empirical evidence and actionable insights for enhancing temporal coherence—establishing a new paradigm for reliability evaluation and optimization of world models.

To address catastrophic forgetting in continual physical dynamics modeling under nonstationary visual environments, this paper proposes a forgetting-free Mixture-of-Gaussians World Model (MG-WM). MG-WM integrates predictive experience replay with an exploration-conservative dual-value estimation mechanism, enabling stable visual dynamics modeling and policy optimization amid environmental evolution. It is the first method to unify three critical properties—environmental simulation continuity, predictive guidance of experience replay, and exploration-conservative balance in behavioral policies—within a model-based reinforcement learning (MBRL) framework. Evaluated on the DeepMind Control and Meta-World continual control benchmarks, MG-WM significantly outperforms state-of-the-art methods. Moreover, on evolving-domain video prediction tasks, it effectively mitigates spatiotemporal dynamics forgetting, demonstrating robust adaptation to dynamic distribution shifts.

Existing latent world models rely on single-step prediction, leading to error accumulation during recursive rollout in long-horizon planning and suffering from a mismatch between their training objective and the actual planning task. This work proposes the Variable-Length World Model (VLWM), which introduces, for the first time, a mechanism for predicting future latent states based on variable-length action sequences. VLWM employs a curriculum learning strategy that progressively optimizes the model from short- to long-horizon predictions and is accompanied by a tailored latent-space planning algorithm. Evaluated across multiple long-horizon control tasks, VLWM outperforms the current state-of-the-art method, LeWM, by an average of 13%, with particularly pronounced gains in tasks requiring extended planning horizons.

Current world models suffer from limited generalizability, lack of interactive capability, and insufficient long-horizon consistency, while video generation models lack causal control and action conditioning. This paper introduces GLP—a General, Interactive, and Long-horizon world model architecture—capable of high-fidelity, action-conditioned, video-level future state simulation in open domains. GLP unifies language instructions and visual dynamics within a shared latent space, integrating an autoregressive latent dynamics backbone driven by a large language model with a video diffusion decoder, trained on large-scale video–action pairs. Experiments demonstrate that GLP significantly outperforms existing methods on action-conditioned simulation, hundred-frame-long horizon prediction, and simulation-based reasoning tasks, exhibiting strong generalization and practical utility.

This work addresses the challenge of unifying prediction, planning, and irreversibility within world models. It formulates prediction as a probability measure over future trajectories and, under a local Markov assumption, employs the Onsager–Machlup action functional to decompose latent dynamics into reversible and irreversible components in path space. The authors introduce rollout-based entropy production as an operational measure of irreversibility. Through path-integral analysis, attention mechanism inspection, and small-scale model experiments, they find that attention asymmetry emerges in response to increasing data irreversibility. While symmetrization interventions suppress entropy production, they selectively impair long-horizon prediction of irreversible processes yet preserve the model’s capacity to capture relaxation dynamics.

This work addresses a critical limitation in existing latent-variable world models, which rely on average prediction error over training data for training and selection—a metric that fails to reflect actual controller performance due to a mismatch between the evaluation distribution and the distribution queried by the planner. The authors propose instead to center model assessment on the discrepancy between predicted and true costs over states reachable by the planner. They establish, for the first time, a rigorous theoretical link between this discrepancy and control suboptimality, proving it provides a valid upper bound on performance loss, whereas conventional prediction errors neither bound nor track performance. Leveraging control theory, spectral analysis, and non-normal operator theory, they decompose the discrepancy into an intrinsic manifold residual and an off-manifold divergence term, and introduce a fidelity score to quantify alignment of the planner’s reachable distribution. Experiments on synthetic systems and model predictive control confirm that the proposed metric reliably tracks control performance, while single-step prediction error shows virtually no correlation.

This work aims to bridge the gap between stateless video generation models and classical state-centric world model theories. To this end, it proposes a dual-pillar conceptual framework centered on state construction and dynamics modeling. In state construction, the framework integrates implicit approaches (context management) with explicit ones (latent compression); in dynamics modeling, it combines knowledge infusion with architectural reconfiguration. The study advocates shifting evaluation criteria from visual fidelity toward functional benchmarks, highlighting persistence and causality as two critical frontiers. By establishing a theoretical foundation for building general-purpose world simulators endowed with physical persistence and causal reasoning capabilities, this work advances the evolution of video generation models toward functional world models.