AnimaMimic: Imitating 3D Animation from Video Priors

2025-12-16

AI Summary

Existing methods struggle to simultaneously achieve motion realism, physical plausibility, and compatibility with standard animation pipelines. To address this, we propose a novel framework for automatically converting static 3D meshes into physically plausible, temporally coherent, and artist-editable 3D animations. Our approach is the first to incorporate motion priors from video diffusion models into the 3D rigging process: leveraging supervision from monocular animated videos, we jointly optimize automatic skeleton generation, skinning weight estimation, and joint parameter learning. We further integrate a differentiable soft-tissue dynamics simulator to enforce physically grounded mesh deformations, and employ differentiable rendering for end-to-end training. The resulting animations exhibit high temporal coherence, natural motion quality, and standardized skeletal structures—enabling seamless integration into industrial animation pipelines and significantly lowering the barrier to high-fidelity 3D animation production.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Creating realistic 3D animation remains a time-consuming and expertise-dependent process, requiring manual rigging, keyframing, and fine-tuning of complex motions. Meanwhile, video diffusion models have recently demonstrated remarkable motion imagination in 2D, generating dynamic and visually coherent motion from text or image prompts. However, their results lack explicit 3D structure and cannot be directly used for animation or simulation. We present AnimaMimic, a framework that animates static 3D meshes using motion priors learned from video diffusion models. Starting from an input mesh, AnimaMimic synthesizes a monocular animation video, automatically constructs a skeleton with skinning weights, and refines joint parameters through differentiable rendering and video-based supervision. To further enhance realism, we integrate a differentiable simulation module that refines mesh deformation through physically grounded soft-tissue dynamics. Our method bridges the creativity of video diffusion and the structural control of 3D rigged animation, producing physically plausible, temporally coherent, and artist-editable motion sequences that integrate seamlessly into standard animation pipelines. Our project page is at: https://xpandora.github.io/AnimaMimic/

Problem

• Automates 3D animation creation from static meshes
• Transfers 2D video motion priors to 3D skeletal animation
• Enhances realism with physics-based soft-tissue simulation

Innovation

• Uses video diffusion models for motion priors
• Automatically constructs skeleton and skinning weights
• Integrates differentiable simulation for physical realism