Real-Time Physics Simulation with Dynamic Mesh-Gaussian Reconstructions

2026-05-29

AI Summary

This work addresses the challenge of efficiently integrating dynamic 3D reconstruction with physics simulation, which is hindered by the complexity of collision detection under changing mesh topologies. The authors propose a dual-representation framework that employs a fixed-topology mesh to enable efficient physical simulation while leveraging Gaussian splatting for high-quality rendering. To handle topological changes, they introduce strategies including vertex buffer updates, temporal correspondence tracking, and stencil projection. Their systematic evaluation—the first of its kind—demonstrates a 4.65× speedup in simulation compared to variable-topology baselines, albeit at the cost of a 65–80% reduction in geometric fidelity during topology transitions. These findings reveal a fundamental trade-off between high-fidelity reconstruction and physics-compatible mesh topologies.

Abstract

Integrating dynamic 3D reconstructions into physics simulation requires fixed mesh topology for efficient collision detection, but state-of-the-art methods like DG-Mesh produce varying topology optimized for geometric quality. We investigate whether topology conversion can enable physics integration while preserving reconstruction fidelity. We propose a dual-representation framework combining fixed-topology meshes for physics with Gaussian splatting for rendering, achieving 4.65$\times$ speedup over varying-topology baselines through runtime vertex buffer updates. We evaluate two conversion strategies, temporal correspondence tracking and template-based projection, against native fixed-topology methods (MaGS) on the DG-Mesh dataset. Our evaluation reveals that both conversion approaches incur 65-80% geometric degradation, producing results inferior to MaGS despite DG-Mesh's superior initial quality. This demonstrates that high-quality reconstruction and physics-compatible topology represent fundamentally distinct objectives that cannot be reconciled through post-processing. Our findings inform future development of physics-aware reconstruction methods and our framework enables real-time simulation with any fixed-topology approach.

Problem

• physics simulation
• dynamic 3D reconstruction
• mesh topology
• collision detection
• real-time simulation

Innovation

• dynamic reconstruction
• fixed-topology mesh
• Gaussian splatting
• physics simulation
• topology conversion