GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator

2024-12-23

arXiv.org

AI Summary

This work addresses the challenge of high-fidelity, efficient simulation of dynamic behaviors of real-world elastic objects. Methodologically, we propose a continuum-mechanics-based neural simulation framework that employs Gaussian kernels as fundamental continuous material units and introduces a Center-of-Mass System (CMS) hierarchical architecture—explicitly embedding physical constraints such as mass and momentum conservation to enable interpretable, physics-consistent modeling across coarse-to-fine granularities. Our key contributions are: (i) the first integration of Gaussian kernels into continuum-based elastic modeling, and (ii) the introduction of an explicit physics-constrained CMS hierarchical simulation paradigm. Evaluated on our newly established READY benchmark—a real-world video dataset—the method significantly outperforms existing physics-driven approaches in dynamic simulation accuracy. Both source code and trained models will be made publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In this work, we introduce GauSim, a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels. Unlike traditional methods that treat kernels as particles within particle-based simulations, we leverage continuum mechanics, modeling each kernel as a continuous piece of matter to account for realistic deformations without idealized assumptions. To improve computational efficiency and fidelity, we employ a hierarchical structure that organizes kernels into Center of Mass Systems (CMS) with explicit formulations, enabling a coarse-to-fine simulation approach. This structure significantly reduces computational overhead while preserving detailed dynamics. In addition, GauSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations. To validate our approach, we present a new dataset, READY, containing multi-view videos of real-world elastic deformations. Experimental results demonstrate that GauSim achieves superior performance compared to existing physics-driven baselines, offering a practical and accurate solution for simulating complex dynamic behaviors. Code and model will be released. Project page: https://www.mmlab-ntu.com/project/gausim/index.html .

Problem

• Simulate dynamic behaviors of elastic objects using Gaussian kernels.
• Improve computational efficiency with hierarchical CMS structure.
• Ensure physically plausible simulations with explicit physics constraints.

Innovation

• Neural network-based simulator using Gaussian kernels
• Hierarchical structure for efficient coarse-to-fine simulation
• Explicit physics constraints ensure interpretable, robust results