ReFM: Semantic-Aware Refinement Flow Model for Motion Retargeting

2026-09-25

AI Summary

This study addresses the challenge of learning semantics for motion retargeting without high-quality paired data, as well as the bottleneck whereby a single regression objective struggles to jointly satisfy physical and semantic constraints. To this end, we propose a source-mesh-agnostic, energy-guided model. Methodologically, retargeting is reformulated as a progressive refinement pipeline compatible with diverse initialization strategies. An SO(3) normalizer and a contrastive cross-character semantic encoder are introduced to enforce semantic consistency without paired supervision. Furthermore, the framework integrates flow matching, inverse kinematics, and energy function optimization. By effectively preserving semantic intent while ensuring physical plausibility, the proposed approach significantly outperforms conventional direct regression methods, yielding substantial improvements in overall retargeting quality.

Abstract

Motion retargeting transfers motion across characters with different skeletal structures while preserving semantic intent and physical plausibility. Despite recent progress, two fundamental questions remain: (i) how can reliable source-motion semantics be learned without high-quality paired retargeting data, and (ii) how should retargeting be formulated when no reliable paired motion can serve as a definitive regression objective? Existing methods commonly preserve semantics by constraining predictions toward copied motions. However, such initializations entangle useful articulation cues with artifacts caused by mismatched skeletal proportions and body geometry. Moreover, directly regressing a final motion in one forward pass is restrictive because retargeting is inherently underdetermined, and the desired solution must balance semantic fidelity with target-specific physical and temporal constraints rather than match a unique paired target. Motivated by these limitations, we propose ReFM, a source-mesh-agnostic, energy-guided model that reformulates motion retargeting as progressive refinement. First, an SO(3) canonicalizer removes redundant global-orientation variations. Second, a cross-character semantic encoder, pretrained through contrastive learning, provides a character-invariant representation for both optimization guidance and semantic evaluation. ReFM then progressively refines an initialized target motion through a learned flow guided by semantic consistency, physical plausibility, temporal coherence, and minimal motion modification. The framework is compatible with different initialization strategies, including both direct motion copying and Autodesk HumanIK, an industry-standard full-body inverse-kinematics retargeting system.

Problem

• Motion Retargeting
• Semantic Preservation
• Paired Data Scarcity
• Underdetermined Problem

Innovation

• Motion Retargeting
• Progressive Refinement
• Contrastive Learning
• Flow Model
• Energy-Guided Optimization