Gaussian-Augmented Physics Simulation and System Identification with Complex Colliders

2025-11-10

AI Summary

Existing video-based system identification methods struggle to model complex physical interactions involving collisions between objects and non-planar rigid bodies, primarily due to oversimplified assumptions of planar collision geometries. To address this, we propose AS-DiffMPM—a novel framework that for the first time integrates a differentiable collision handling mechanism into the Material Point Method (MPM) physics simulator, enabling end-to-end joint optimization with arbitrarily shaped colliders. Our approach combines Gaussian-enhanced rendering with a new view synthesis technique to jointly invert physical properties—including geometry, appearance, elasticity, and friction—from multi-view visual observations. Evaluated on challenging collision scenarios featuring non-planar contacts and multi-angle views, AS-DiffMPM achieves significantly improved parameter estimation accuracy and generalization capability. Moreover, it is compatible with diverse view synthesis paradigms, thereby extending the applicability of video-based physical system identification to realistic, geometrically complex environments.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: State Estimation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

System identification involving the geometry, appearance, and physical properties from video observations is a challenging task with applications in robotics and graphics. Recent approaches have relied on fully differentiable Material Point Method (MPM) and rendering for simultaneous optimization of these properties. However, they are limited to simplified object-environment interactions with planar colliders and fail in more challenging scenarios where objects collide with non-planar surfaces. We propose AS-DiffMPM, a differentiable MPM framework that enables physical property estimation with arbitrarily shaped colliders. Our approach extends existing methods by incorporating a differentiable collision handling mechanism, allowing the target object to interact with complex rigid bodies while maintaining end-to-end optimization. We show AS-DiffMPM can be easily interfaced with various novel view synthesis methods as a framework for system identification from visual observations.

Problem

• Estimating physical properties with arbitrarily shaped colliders
• Handling object interactions with complex non-planar surfaces
• Enabling system identification from visual observations

Innovation

• Differentiable MPM framework with arbitrary shaped colliders
• Incorporates differentiable collision handling mechanism
• Enables object interaction with complex rigid bodies