Virtual Community: An Open World for Humans, Robots, and Society
🤖 AI Summary
This work addresses embodied social intelligence in open-world human-robot coexistence, focusing on cooperation, competition, and harmonious cohabitation. To this end, we introduce a scalable virtual community platform grounded in realistic 3D environments and a general-purpose physics engine, enabling large-scale human–robot–society interaction modeling. We formally define two novel challenges—“community planning” and “community robotics”—to unify high-level multi-agent task reasoning with heterogeneous robot coordination control. Methodologically, our approach integrates an open-source multi-agent simulator, a real-world-aligned community generation pipeline, and joint modeling of outdoor/indoor spatial layouts with role-aware embodied agents. Extensive evaluation across multiple benchmarks demonstrates the platform’s effectiveness in open-world task planning and low-level collaborative control, significantly improving consistency and scalability in socially grounded generation and task-driven interaction.
Application Category
📝 Abstract
The rapid progress in AI and Robotics may lead to a profound societal transformation, as humans and robots begin to coexist within shared communities, introducing both opportunities and challenges. To explore this future, we present Virtual Community-an open-world platform for humans, robots, and society-built on a universal physics engine and grounded in real-world 3D scenes. With Virtual Community, we aim to study embodied social intelligence at scale: 1) How robots can intelligently cooperate or compete; 2) How humans develop social relations and build community; 3) More importantly, how intelligent robots and humans can co-exist in an open world. To support these, Virtual Community features: 1) An open-source multi-agent physics simulator that supports robots, humans, and their interactions within a society; 2) A large-scale, real-world aligned community generation pipeline, including vast outdoor space, diverse indoor scenes, and a community of grounded agents with rich characters and appearances. Leveraging Virtual Community, we propose two novel challenges. The Community Planning Challenge evaluates multi-agent reasoning and planning ability in open-world settings, such as cooperating to help agents with daily activities and efficiently connecting other agents. The Community Robot Challenge requires multiple heterogeneous robots to collaborate in solving complex open-world tasks. We evaluate various baselines on these tasks and demonstrate the challenges in both high-level open-world task planning and low-level cooperation controls. We hope that Virtual Community will unlock further study of human-robot coexistence within open-world environments.
Problem
Research questions and friction points this paper is trying to address.
Studying human-robot social intelligence in open-world environments
Developing multi-agent cooperation and competition in shared communities
Creating simulation platform for human-robot coexistence challenges
Innovation
Methods, ideas, or system contributions that make the work stand out.
Open-world platform with physics engine
Multi-agent simulator for human-robot interactions
Real-world aligned community generation pipeline
🔎 Similar Papers
No similar papers found.
