Mental World Modeling

2026-07-29

AI Summary

This work addresses the limitation of existing world models, which focus solely on physical states and thus struggle to accurately predict human-driven behaviors. To overcome this, the paper introduces the first world modeling framework that explicitly incorporates mental variables—such as beliefs and intentions—into its core architecture. It proposes a physics–mind coupled dynamic mechanism to jointly model their interactions and integrates a modular pipeline comprising state parsing, goal observation generation, action decomposition, joint transition modeling, and branching value evaluation, leveraging large language models for interpretable mental-state reasoning. The authors also provide MENTIS, a trainable-free, testable baseline, and demonstrate through experiments on multimodal situated decision-making datasets that explicit modeling of mental states is crucial for behavior prediction, thereby revealing key bottlenecks in current approaches to mental modeling.

Abstract

World models enable a predictive substrate for planning and action, yet existing formulations merely answer a physical question: what/where it is, and how will it evolve. Human behavior, however, is driven by hidden mental state (what a person believes, wants, intends, feels, and considers socially permissible), so a model that tracks the physical scene but not what each agent knows and believes about it predicts the wrong action for the right-looking scene. We formulate Mental World Modeling (MWM), a generic theoretical framework that makes mental variables core components of a world model rather than posthoc rationales: MWM aintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions jointly update both components. We instantiate the framework in MENTIS, a training-free and fully inspectable baseline that decomposes the process into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. On a manually constructed, quality-controlled dataset of situated decision scenarios spanning text, image, and sounding-video stories, experiments with 8 modern LLM-based world models demonstrate that explicitly modeling the mental state is essential for predicting human decisions. Deeper analyses further expose the bottlenecks of current mental world modeling. We expect MWM as a next stage of world modeling, from simulating physical scenes to simulating the minds that act in them.

Problem

• mental state
• world modeling
• human decision prediction
• belief modeling
• physical-mental coupling

Innovation

• Mental World Modeling
• world models
• mental state inference
• human decision prediction
• coupled physical-mental simulation