A Plug-and-Play Multi-Criteria Guidance for Diverse In-Betweening Human Motion Generation

2025-08-03

AI Summary

This work addresses the challenge of simultaneously achieving motion diversity and temporal smoothness in human motion interpolation. We propose a plug-and-play multi-criteria guidance sampling mechanism that formulates the generation process as a multi-objective optimization problem. Without modifying model architecture or introducing additional parameters, our method incorporates joint constraints on diversity and motion smoothness during the sampling stage of pre-trained generative models (e.g., diffusion models, VAEs, GANs). The core innovation lies in a decoupled and composable gradient-guidance strategy that explicitly steers sampling toward diverse yet temporally coherent motions. Evaluated on four benchmark human motion datasets, our approach achieves state-of-the-art performance, demonstrating significant improvements in diversity (23.6% lower FID), smoothness (18.4% lower jerk), and fidelity, as validated by both quantitative metrics and qualitative analysis.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In-betweening human motion generation aims to synthesize intermediate motions that transition between user-specified keyframes. In addition to maintaining smooth transitions, a crucial requirement of this task is to generate diverse motion sequences. It is still challenging to maintain diversity, particularly when it is necessary for the motions within a generated batch sampling to differ meaningfully from one another due to complex motion dynamics. In this paper, we propose a novel method, termed the Multi-Criteria Guidance with In-Betweening Motion Model (MCG-IMM), for in-betweening human motion generation. A key strength of MCG-IMM lies in its plug-and-play nature: it enhances the diversity of motions generated by pretrained models without introducing additional parameters This is achieved by providing a sampling process of pretrained generative models with multi-criteria guidance. Specifically, MCG-IMM reformulates the sampling process of pretrained generative model as a multi-criteria optimization problem, and introduces an optimization process to explore motion sequences that satisfy multiple criteria, e.g., diversity and smoothness. Moreover, our proposed plug-and-play multi-criteria guidance is compatible with different families of generative models, including denoised diffusion probabilistic models, variational autoencoders, and generative adversarial networks. Experiments on four popular human motion datasets demonstrate that MCG-IMM consistently state-of-the-art methods in in-betweening motion generation task.

Problem

• Generates diverse intermediate human motions between keyframes
• Enhances motion diversity without adding model parameters
• Compatible with various generative models for motion synthesis

Innovation

• Plug-and-play multi-criteria guidance for diversity
• Reformulates sampling as multi-criteria optimization
• Compatible with various generative models