Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction

2026-10-08

AI Summary

This study addresses the limitations of existing multi-agent world models in modeling fine-grained interactions and maintaining shared environment consistency by proposing the ME-World framework. This method jointly denoises multi-view flows through a shared token sequence and introduces a shared environment memory mechanism. Furthermore, it pioneers cross-view and state-update consistency constraints under fine-grained actions to enable synchronized first-person video generation for multiple agents. Experimental results demonstrate that ME-World achieves significant improvements in shared world consistency, action control precision, identity preservation, and overall video generation quality. By effectively bridging these gaps, this work fills a critical void in the modeling of multi-agent embodied interactions.

Abstract

Egocentric world models predict first-person observations conditioned on an agent's actions, but most focus on a single agent. Real embodied settings often involve multiple agents that act and interact within a shared environment. Existing multi-agent world models rely on coarse actions like locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. We formulate multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions in a shared world. This requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates. We propose Multi-agent Egocentric World Model (ME-World), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents' target-view poses, and grounds generation with shared environment memory. We train and evaluate on real and synthetic multi-agent data and introduce shared-world consistency metrics for environment, update, and identity consistency. Experiments show ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods.

Problem

• Multi-agent egocentric world model
• Fine-grained embodied interaction
• Synchronized ego-stream generation
• Shared-world consistency

Innovation

• Multi-Agent Egocentric World Model
• Fine-Grained Embodied Interaction
• Joint Denoising
• Shared Environment Memory
• Cross-View Consistency