Morph: A Motion-free Physics Optimization Framework for Human Motion Generation

2024-11-22

arXiv.org

AI Summary

Existing generative methods often neglect physical constraints, resulting in artifacts such as foot sliding and floating, and lack effective physics-based optimization for noisy motion data. This paper proposes a two-stage collaborative optimization framework that operates without ground-truth motion-capture data: first synthesizing large-scale diverse motion sequences, then refining them via physics-based imitation using MuJoCo or PyBullet, with gradient-based backward optimization of the generator. We introduce the first “motion-agnostic” physical optimization paradigm, establishing a closed-loop joint fine-tuning mechanism between generator and refiner, augmented by an adversarial physics projection loss. Evaluated on text- and music-driven dance generation, our method achieves state-of-the-art performance: foot sliding is reduced by 72%, floating frames decrease by 89%, and both physical plausibility and motion quality are significantly enhanced.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Human motion generation plays a vital role in applications such as digital humans and humanoid robot control. However, most existing approaches disregard physics constraints, leading to the frequent production of physically implausible motions with pronounced artifacts such as floating and foot sliding. In this paper, we propose extbf{Morph}, a extbf{Mo}tion-f extbf{r}ee extbf{ph}ysics optimization framework, comprising a Motion Generator and a Motion Physics Refinement module, for enhancing physical plausibility without relying on costly real-world motion data. Specifically, the Motion Generator is responsible for providing large-scale synthetic motion data, while the Motion Physics Refinement Module utilizes these synthetic data to train a motion imitator within a physics simulator, enforcing physical constraints to project the noisy motions into a physically-plausible space. These physically refined motions, in turn, are used to fine-tune the Motion Generator, further enhancing its capability. Experiments on both text-to-motion and music-to-dance generation tasks demonstrate that our framework achieves state-of-the-art motion generation quality while improving physical plausibility drastically.

Problem

• Generating physically plausible human motions without artifacts
• Training physics optimizers with noisy synthetic motion data
• Enhancing motion quality and stability in real applications

Innovation

• Motion-free physics optimization framework
• Synthetic noisy motion data generation
• Physics simulator enforcing physical constraints