Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations

2026-08-02

AI Summary

This work addresses the limitation of conventional motion transfer methods, which fail when significant morphological or articulation differences exist due to their reliance on fixed structural correspondences. To overcome this, we propose a two-stage cross-morphology motion transfer framework: first, multi-granularity abstract motion representations are learned to construct cross-category video pairs; these are then internalized into an end-to-end reference-video-conditioned generative model that synthesizes motion without requiring explicit motion extraction at inference time. Our approach eliminates the need for structural correspondence and, for the first time, enables reference-driven generation without explicit motion representations. We also introduce and release the OpenVMT dataset and benchmark. Experiments demonstrate state-of-the-art performance in motion fidelity and structural consistency across Same, Near, and Far category gaps, confirming the method’s strong generalization and practical utility.

Abstract

Video motion transfer aims to animate a target object using dynamics from a reference video. Existing formulations largely rely on fixed structural correspondence, which becomes ill-defined when reference and target objects differ substantially in morphology, articulation, or deformation mechanisms. We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies. To realize this, we propose a two-stage framework. Stage~I learns complementary multi-granularity abstract motion views and uses them to bootstrap cross-category video pairs that preserve transferable dynamics across diverse morphologies. Stage~II internalizes this supervision into direct reference-video-conditioned generation, removing the need for explicit motion extraction at inference. We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and evaluating image- and text-conditioned motion transfer across Same, Near, and Far category gaps, and plan to release both upon acceptance. Extensive experiments demonstrate state-of-the-art motion fidelity and target preservation. Project page: https://miniz233.github.io/MotionBeyondMorphology/

Problem

• motion transfer
• cross-category
• morphology
• structural correspondence
• video animation

Innovation

• motion transfer
• cross-category
• abstract motion representation
• morphology-agnostic
• video generation