BeliefNest: A Joint Action Simulator for Embodied Agents with Theory of Mind

πŸ“… 2025-05-18
πŸ“ˆ Citations: 0
✨ Influential: 0
πŸ“„ PDF
πŸ€– AI Summary
Embodied agents lack explicit Theory of Mind (ToM) modeling capabilities for open-domain collaborative tasks. Method: We propose BeliefNest, an open-source joint-action simulator built in Minecraft, featuring the first dynamic hierarchical nested belief modeling framework. It structurally represents self- and other-centered multi-order belief states as parseable graph models, directly mapped to LLM prompts to enable interpretable, evaluable ToM-driven decision-making. The approach integrates embodied simulation, hierarchical belief graphs, LLM prompt engineering, and a novel false-belief task evaluation protocol. Contribution/Results: Experiments demonstrate that BeliefNest accurately infers others’ beliefs and predicts belief-guided behavior. Quantitative evaluation on standardized false-belief tasks confirms that nested belief modeling significantly enhances multi-agent coordination performance, establishing a new benchmark for interpretable, ToM-aware embodied AI.

Technology Category

Multiagent Systems: Modeling other AgentsCognitive Modeling & Cognitive Systems: Simulating Human BehaviorKnowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSearch and Retrieval-Augmented AI: Agentic search
πŸ“ Abstract
This paper introduces an open-source simulator, BeliefNest, designed to enable embodied agents to perform collaborative tasks by leveraging Theory of Mind. BeliefNest dynamically and hierarchically constructs simulators within a Minecraft environment, allowing agents to explicitly represent nested belief states about themselves and others. This enables agent control in open-domain tasks that require Theory of Mind reasoning. The simulator provides a prompt generation mechanism based on each belief state, facilitating the design and evaluation of methods for agent control utilizing large language models (LLMs). We demonstrate through experiments that agents can infer others' beliefs and predict their belief-based actions in false-belief tasks.
Problem

Research questions and friction points this paper is trying to address.

Simulate collaborative tasks for embodied agents using Theory of Mind
Model nested belief states in open-domain tasks dynamically
Enable belief-based action prediction in false-belief scenarios
Innovation

Methods, ideas, or system contributions that make the work stand out.

Dynamic hierarchical simulators in Minecraft
Explicit nested belief states representation
Prompt generation for LLM-based agent control
πŸ”Ž Similar Papers
No similar papers found.
πŸ’Ό Related Jobs
No related jobs found.
R
Rikunari Sagara
University of Shizuoka
K
Koichiro Terao
Okayama Prefectural University
Naoto Iwahashi
Naoto Iwahashi
Okayama Prefectural University
Artificial Intelligence