AIM: Adaptive Interaction Modeling Networks for Real-to-Sim Soft-Body Simulation

2026-10-05

AI Summary

This study addresses the issues of cumulative errors caused by neighborhood misjudgment and limited generalization in soft body simulation by proposing an adaptive interaction modeling framework. The framework formulates simulation as a local-global interaction process, employing an adaptive graph neural network that fuses motion history with geometric information to dynamically update particle relationships. This local modeling is combined with a unified control point interface for global coordination, and the model is trained under multi-step autoregressive supervision. Experimental results demonstrate that the proposed method reduces predictive tracking error by 20.0% and long-horizon error by 22.8%, while achieving zero-shot transferability across unseen actions, instances, and scenes.

Abstract

Deformable-object manipulation is essential for robotic tasks such as folding laundry and handling food, where robots must control shape changes as well as object motion. Predictive soft-body simulation supports these tasks by anticipating deformation under external interactions. However, spatial neighborhoods can misrepresent deformation dependencies, introducing local errors that accumulate over successive predictions. Models fitted to individual scenes must also accommodate changes in object geometry and manipulation conditions. In this work, we propose AIM, an Adaptive Interaction Modeling framework that treats real-to-sim soft-body simulation as a local-global interaction modeling problem. AIM uses motion history and geometry to adapt particle relations over current spatial neighbors and retained connections, while geometry-conditioned global communication coordinates object-wide responses. A unified kinematic control-point interface represents different manipulation configurations, and multi-step autoregressive supervision trains the model on its own predicted trajectories. Experiments on PhysTwin and PGND demonstrate improved motion accuracy and visual fidelity, with a 20.0% reduction in future-prediction tracking error relative to PhysTwin and a 22.8% reduction in mean long-horizon particle error across six object categories relative to PGND. The framework further supports transfer across actions, object instances, and scenes, including zero-shot transfer from robot interactions to human manipulation without target-domain dynamics fitting.

Problem

• soft-body simulation
• deformable-object manipulation
• error accumulation
• generalization
• real-to-sim transfer

Innovation

• Adaptive Interaction Modeling
• Soft-body Simulation
• Local-global Interaction
• Autoregressive Supervision
• Zero-shot Transfer