Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

2026-09-30

AI Summary

This work addresses the limitation of existing VLM agents that rely solely on prospective prediction, lacking causal consistency verification and physical coherence. We introduce a retrospective world modeling paradigm that quantifies the causal contribution of actions to state transitions via probabilistic attribution estimation. This approach constructs self-consistent reward signals (SCR) to provide dense feedback for reinforcement learning, guiding the generation of physically grounded behaviors. By overcoming the constraints of traditional prospective reasoning, our method enables causal consistency verification between actions and states, substantially enhancing policy robustness and generalization capabilities. Experimental results demonstrate that the proposed framework outperforms purely prospective modeling baselines, establishing retrospective causal verification as an effective mechanism for improving embodied agent performance in complex environments.

Abstract

Equipping VLM agents with world modeling capabilities has shown strong potential for complex reasoning and long-horizon planning, while reducing the dependence of policy learning on costly real-world interactions. Existing methods mainly rely on prospective simulation to predict the consequences of candidate actions. However, this forward-only paradigm focuses on what will happen next and provides limited constraints for verifying whether an action is causally consistent with the observed state transition, which can lead to plausible-looking but physically incoherent behaviors. In this paper, we challenge the view of world modeling as only prospective prediction and introduce Retrospective World Modeling, a new agent learning paradigm that enables agents to reason backward by estimating the retrospective attribution distribution $P(\hat{a}{t}|s_t, s{t+1})$ for the action that most likely caused a given transition. Based on this capability, we formulate the Self-Consistency Reward (SCR), an intrinsic signal that measures the probabilistic consistency between the policy action and the retrospective explanation. Integrating SCR into reinforcement learning provides dense transition-level feedback and steers agents toward behaviors that are both task-effective and physically grounded. Extensive experiments across diverse agentic tasks show that our method substantially improves policy robustness and generalization over prospective-only world modeling baselines.

Problem

• Vision-Language Model Agents
• World Modeling
• Causal Consistency
• Physical Coherence
• Prospective Prediction

Innovation

• Retrospective World Modeling
• Self-Consistency Reward
• VLM Agents
• Reinforcement Learning
• Causal Consistency