BoxTwin: Learning Elastoplastic Articulated Object Dynamics from Videos

2026-07-19

AI Summary

Existing digital twin approaches struggle to accurately model the dynamic behavior of elasto-plastic articulated objects (EAOs) exhibiting nonlinear elasticity, plastic yielding, and damage accumulation. This work proposes BoxTwin, a novel framework that, for the first time, integrates video-driven 3D scene reconstruction with physics-aware identification of elasto-plastic constitutive models to enable precise prediction of plastic deformations in EAOs after prolonged interaction. By jointly optimizing geometric reconstruction, constitutive parameter identification, and dynamics simulation, BoxTwin effectively reproduces contact-induced plastic deformations and accurately tracks joint trajectories in both manual folding and dual-arm manipulation experiments. The method substantially enhances the capability of digital twins to model and control deformable articulated objects in unstructured environments.

Abstract

Digital twins enable robots to anticipate and adapt to physical interactions, but existing models struggle with elastoplastic articulated objects (EAOs) that exhibit nonlinear elasticity, plastic yielding, and damage accumulation. We present BoxTwin, an interactive digital twin framework that learns the full dynamics of EAOs from videos. Our pipeline reconstructs the scene, identifies a physics aware constitutive model for each EAO. Experiments on manual folding and dual arm manipulation of EAOs show that BoxTwin accurately tracks joint trajectories and reproduces post contact plastic behavior over long horizons. By integrating video driven reconstruction with elastoplastic damage modeling, BoxTwin advances digital twins toward predictive, adaptive control of deformable articulated objects in unstructured environments.

Problem

• elastoplastic articulated objects
• digital twins
• nonlinear elasticity
• plastic yielding
• damage accumulation

Innovation

• digital twin
• elastoplastic dynamics
• articulated objects
• video-based reconstruction
• constitutive modeling