procedural animation

Existing surveys predominantly focus on isolated methodologies, lacking a systematic examination of the end-to-end pipeline for human motion video generation. To address this gap, we propose the first unified framework encompassing five core stages: input specification, motion planning, generation, optimization, and output rendering—supporting over ten subtasks driven by visual, textual, and audio modalities. Our work systematically reviews 200+ papers and constructs the field’s first comprehensive technical taxonomy. We innovatively investigate the potential of large language models (LLMs) for motion semantic modeling and cross-modal alignment. Furthermore, we integrate state-of-the-art techniques—including diffusion models, generative adversarial networks (GANs), and multimodal fusion—to identify key breakthroughs and release an open-source model library. This study fills a critical void in holistic, cross-cutting research on human motion video generation, providing both theoretical foundations and practical guidelines for applications such as digital avatars.

This paper presents a systematic review of recent advances in human motion prediction, reconstruction, and generation. Addressing key challenges—including instability in long-horizon prediction, limited reconstruction accuracy, and insufficient physical plausibility and diversity in motion generation—we propose a unified “prediction–reconstruction–generation” co-evolutionary framework. Our method integrates diffusion models with physics-informed dynamical constraints in the loss function to enhance motion realism and biomechanical consistency. Furthermore, we introduce multimodal alignment and fine-grained contextual modeling to improve text-to-motion generation and human-object interaction synthesis. Extensive experiments demonstrate significant improvements over state-of-the-art methods: a 23% reduction in average prediction error for long-horizon motion forecasting, an 18% decrease in MPJPE for 3D pose reconstruction, and a 31% reduction in FID score for motion generation. The framework supports applications in digital avatars, embodied AI, and real-time AR interaction.

Generating long-horizon, multi-action coherent human motions in 3D scenes remains challenging due to drift accumulation, action discontinuity, and poor scene adaptability. Method: We propose an animator-guided, scene-agnostic iterative generation framework. It establishes a target-centric canonical coordinate system to decouple path planning from motion transition, and employs motion decomposition modeling with coordinate-system reparameterization—enabling zero-shot deployment on pure motion-capture data without scene-aware annotations or fine-tuning. Contribution/Results: To our knowledge, this is the first method to generate drift-free, chained multi-action sequences (e.g., “grasp → sit → lean”) in diverse real-world scanned environments—including HPS, Replica, Matterport, and ScanNet—using only sparse keypoint constraints and a seed motion. Unlike existing 3D navigation approaches, ours requires no scene rendering, geometric encoding, or environment-specific training, achieving superior generalization, motion plausibility, and scene compatibility.

This study addresses the limitations of traditional physics simulators in robotics—such as restricted expressiveness due to simplifying assumptions, high data costs, and difficulties in modeling complex physical interactions—by systematically reviewing video generation models as embodied world models. Integrating high-fidelity, multimodal-conditioned video synthesis with imitation learning, reinforcement learning, and visual planning frameworks, this work provides the first comprehensive analysis of their potential and limitations in tasks including action prediction, dynamics modeling, and policy evaluation. The review highlights breakthroughs in high-fidelity modeling of physical interactions while identifying key challenges in instruction following, physical consistency, and safety. These insights lay a theoretical foundation and outline future directions for replacing conventional simulators and enabling deployment in safety-critical scenarios.

Existing motion generation models often neglect individual body shape variations, relying instead on a generic average human template—leading to physically implausible and anatomically homogeneous motions. To address this, we propose the first generative motion model conditioned on 3D body shape (parameterized by SMPL-X), enabling body-aware motion synthesis without requiring paired motion-capture data. Our approach jointly models the coupling between body shape and motion dynamics, incorporating a cycle-consistency loss, physics-based constraints derived from kinematics and dynamics, and stability regularization. Quantitative evaluation demonstrates consistent superiority over state-of-the-art methods across standard metrics—including FID, Jitter, and Diversity. Qualitative analysis further confirms anatomical plausibility, inertial consistency, and strong cross-body-shape generalization. Overall, our method significantly enhances both the physical realism and inter-individual diversity of synthesized human motion.

Existing approaches to human motion generation often suffer from physically implausible results due to contact modeling limited to the hands. This work proposes a physics-aware framework that explicitly models the full spectrum of contacts—including interactions between the body and objects, scenes, and self-limb collisions—through a continuous distance-driven force model. By integrating soft physical constraints with force and torque balance mechanisms, the method synthesizes multi-body dynamics-consistent motions. It supports arbitrary surface interactions with both static environments and dynamic objects, significantly enhancing the physical plausibility of generated actions. The approach demonstrates strong generalization in complex, dynamic scenarios and establishes a new benchmark for physically consistent human motion generation.

Current image-to-video generation models struggle to accurately simulate mechanical motion governed by kinematic and geometric constraints, often exhibiting inconsistencies in rigidity preservation, component contact, and motion transmission. This work proposes MechVerse—the first benchmark dataset specifically designed for mechanical assembly scenarios—which systematically defines and quantifies mechanical motion consistency in video generation. The benchmark encompasses three levels of mechanism complexity and establishes a multi-tiered evaluation framework integrating synthetic data, structured prompts, standard video metrics, instruction-following scores, and human assessments of motion correctness. Experiments reveal that while state-of-the-art models maintain visual fidelity and temporal smoothness, they perform poorly in terms of mechanical plausibility, with error rates rising significantly as coupling complexity increases.

Existing motion capture datasets suffer from limited diversity, which constrains the generalization capabilities of generative models on rare, highly dynamic, and compositionally complex actions. To address this limitation, this work proposes a method that leverages large-scale synthetic human motion data combined with physics-based plausibility constraints to jointly expand both the training distribution and the size of the discrete codebook. By reconstructing the VQ-VAE motion tokenizer beyond the confines of real-data distributions, the approach substantially broadens the coverage and compositional capacity of the discrete motion representation space. This leads to consistent performance gains in text-to-motion generation and motion in-betweening tasks, and the enhanced representations can be seamlessly integrated into existing frameworks such as MotionGPT, demonstrating both the effectiveness and generalizability of the proposed representation expansion.

This work addresses the challenge that existing methods struggle to accurately interpret diverse motion categories in textual prompts, limiting the quality of multi-instance video synthesis. The authors propose a training-free motion decomposition framework that disentangles complex motion into three canonical types: static, rigid, and non-rigid. Adopting a “plan-then-generate” paradigm, the approach first infers instance-level shape and positional dynamics through a motion graph during the planning phase, then modulates each motion type in a disentangled manner during generation. This method achieves, for the first time, training-free disentanglement of motion categories, introduces motion graph–guided structured semantic representations, and incorporates model-agnostic modules compatible with various diffusion architectures. Experiments demonstrate significant improvements in motion synthesis quality on real-world benchmarks, effectively enabling compositional video generation with multiple instances, appearances, and motion types.

Existing modeling approaches for high-degree-of-freedom articulated objects (e.g., robots) rely heavily on manual annotations or motion sequences, suffering from poor scalability and labor-intensive data curation. Method: This paper introduces the first end-to-end, open-vocabulary (RGB image or text prompt) automatic modeling framework. It jointly performs topology inference via Monte Carlo Tree Search (MCTS) and geometry-driven optimization for joint parameter estimation—requiring neither motion data nor hand-crafted datasets. Contribution/Results: Our method is the first to synthesize physically consistent and functionally plausible articulated models directly from a single RGB image or natural language description. By decoupling structural inference from parametric estimation, it ensures both topological correctness and kinematic plausibility. Evaluated on synthetic and real-world benchmarks, it achieves significant improvements in registration accuracy (+12.3%) and topology recognition accuracy (+18.7%), demonstrating strong generalization and practical utility.