QuadVerse: An Integrated Framework Aligning Visual-Physical Reality for Quadruped Simulation

2026-06-05

AI Summary

This work addresses the sim-to-real gap in quadrupedal robot simulation caused by the coupling of visual, dynamic, and actuator errors by introducing QuadVerse, a unified framework. QuadVerse integrates geometry-constrained 3D Gaussian splatting for scene reconstruction, contact calibration, and actuator residual compensation within an end-to-end pipeline. It synchronously aligns visual perception, physical interaction, and actuator dynamics using RGB video and employs trajectory replay to train a residual dynamics compensator. Experimental results demonstrate that QuadVerse significantly improves scene reconstruction fidelity and motion tracking accuracy, enabling zero-shot deployment of robust vision-based navigation policies without requiring fine-tuning in real-world environments.

Abstract

Simulation is central to robot learning, yet the sim-to-real gap remains a major bottleneck.Existing approaches often tackle visual or dynamic gaps separately, overlooking how these individual mismatches accumulate and propagate throughout the robot's state evolution.In this paper, we introduce QuadVerse, an integrated framework that uses reconstructed scenes as a calibration substrate for aligning visual perception, physical interaction, and actuator dynamics.From captured RGB videos, we reconstruct geometry-constrained 3D Gaussian Splatting (3DGS) scenes that support batched photorealistic ego-view rendering and collision-ready semantic mesh extraction. The meshes further enable contact calibration by initializing spatially varying friction priors and refining them through trajectory-based posterior search.To address remaining actuator discrepancies, QuadVerse trains a residual dynamics compensator by replaying real-world trajectories on the contact-calibrated terrain, reducing the entanglement between terrain-induced contact errors and actuator non-idealities.Experiments show that QuadVerse improves reconstruction quality and locomotion tracking over relevant baselines.Leveraging this foundation, we demonstrate robust zero-shot visual-navigation policy deployment without task-specific real-world rollouts.

Problem

• sim-to-real gap
• quadruped simulation
• visual-physical alignment
• robot learning
• contact dynamics

Innovation

• 3D Gaussian Splatting
• contact calibration
• residual dynamics compensator
• sim-to-real transfer
• zero-shot policy deployment