Technology Category
Computer Vision: Low Level & Physics-based VisionKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal ReasoningIntelligent Robots: Multimodal Perception & Sensor Fusion
DRAWER: Digital Reconstruction and Articulation With Environment Realism
🤖 AI Summary
This work addresses the challenge of reconstructing monocular indoor videos into high-fidelity, interactive digital twin environments—where static scene representations hinder physics-based simulation and real-time interaction. To this end, we propose a dual-scene representation framework that jointly models geometry and appearance via a hybrid neural radiance field (NeRF) and explicit mesh representation. We further introduce a novel semantic-geometric articulation module for movable parts, which detects hinges and reconstructs physically simulatable shapes to enable articulated motion simulation. Our method is end-to-end integrated with neural rendering and deployed within Unreal Engine and robotics simulation platforms. Experiments demonstrate a 32% reduction in geometric reconstruction error and interactive response latency under 16 ms. To our knowledge, this is the first approach to achieve a closed-loop pipeline from real-world capture → simulation training → real-world deployment, successfully generating game-grade interactive digital environments.
Technology Category
Application Category
📝 Abstract
Creating virtual digital replicas from real-world data unlocks significant potential across domains like gaming and robotics. In this paper, we present DRAWER, a novel framework that converts a video of a static indoor scene into a photorealistic and interactive digital environment. Our approach centers on two main contributions: (i) a reconstruction module based on a dual scene representation that reconstructs the scene with fine-grained geometric details, and (ii) an articulation module that identifies articulation types and hinge positions, reconstructs simulatable shapes and appearances and integrates them into the scene. The resulting virtual environment is photorealistic, interactive, and runs in real time, with compatibility for game engines and robotic simulation platforms. We demonstrate the potential of DRAWER by using it to automatically create an interactive game in Unreal Engine and to enable real-to-sim-to-real transfer for robotics applications.
Problem
Research questions and friction points this paper is trying to address.
Converts video of static indoor scenes into interactive digital environments
Reconstructs scenes with fine-grained geometric details and simulatable components
Enables real-time photorealistic environments for gaming and robotics simulations
Innovation
Methods, ideas, or system contributions that make the work stand out.
Dual scene representation for detailed reconstruction
Articulation module for simulatable shapes integration
Real-time photorealistic interactive virtual environments
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
