motion knowledge translation

This work addresses the limited generalization of trajectory prediction models in cross-dataset scenarios, primarily caused by discrepancies in scene layouts, agent behaviors, and perceptual conditions. The authors propose a transferability assessment framework based on latent scene embeddings and distributional distance metrics, establishing the first large-scale transfer experiment suite encompassing 24 mainstream trajectory datasets. Their analysis reveals a strong correlation between inter-dataset similarity and model transfer performance, enabling the design of a transferability scoring metric that effectively predicts cross-domain model behavior. This metric provides both theoretical grounding and practical guidance for pretraining strategies, dataset selection, and the development of foundational models for trajectory prediction.

This work investigates the cross-context transferability of action recognition models to high-level motion concepts (e.g., “hitting”) in novel scenarios. We propose a Motion Transferability Assessment framework and introduce three benchmark datasets—Syn-TA (synthetic), Kinetics400-TA, and Something-Something-v2-TA (real-world)—to systematically evaluate 13 state-of-the-art models. Leveraging multimodal inputs and controlled-variable analysis, we identify critical bottlenecks in fine-grained action discrimination and temporal reasoning. We further demonstrate that disentangling coarse- and fine-grained motion representations significantly improves generalization. Experiments reveal: (1) severe performance degradation on in-distribution variants (e.g., unseen “hitting-a-person” instances); and (2) while large models excel at spatially dominant tasks, their over-reliance on object and background cues hinders intrinsic motion generalization. This work establishes a new benchmark for robust action understanding and provides an interpretable diagnostic pathway for motion-centric transferability.

MotionGlot addresses the challenges of cross-modal motion generation—namely, inconsistent multi-dimensional action spaces across heterogeneous agents (e.g., quadrupeds and humans), scarcity of high-quality annotated data, and difficulty in text-motion alignment—by adapting large language model (LLM) training paradigms to motion synthesis. To this end, it introduces: (1) a multi-entity motion-space alignment mechanism; (2) a text-motion joint embedding framework with instruction fine-tuning; and (3) the first directionally annotated quadruped locomotion dataset and a large-scale, scenario-aware human motion prompting corpus. Evaluated on six generation tasks, MotionGlot achieves an average 35.3% improvement over prior methods and demonstrates end-to-end deployment on a physical quadruped robot. This work establishes a novel paradigm for text-driven, general-purpose motion generation across diverse embodied agents.

Existing video motion customization methods suffer from an imbalance between semantic alignment and visual modeling: semantic-driven approaches often neglect the spatiotemporal complexity of motion, while purely visual adaptation leads to action semantic ambiguity. To address this, we propose a semantic-visual co-modeling framework that enables few-shot personalized motion generation and arbitrary subject transfer. Our method decouples subject-motion representations at the semantic level within diffusion models, introduces a vision-level motion adapter, and employs an alternating embedding training strategy on the SPV dataset. The framework features a dual-embedding semantic understanding mechanism and a parameter-efficient motion adapter. It achieves significant improvements over state-of-the-art methods on both text-to-video (T2V) and image-to-video (I2V) benchmarks. Additionally, we release MotionBench—a new benchmark encompassing diverse motion patterns—to advance standardized evaluation for video motion generation.

Existing animal motion transfer methods primarily focus on human motions and struggle to preserve species-specific behavioral patterns, leading to unnatural and behaviorally inconsistent results. This paper introduces the first generative framework for cross-species animal motion transfer. Our approach addresses this challenge through two key innovations: (1) a Habit Preservation Module coupled with a class-specific Habit Encoder, which explicitly models and enforces typical behavioral patterns of target species; and (2) a hybrid representation integrating skeletal binding with large language model (LLM)-guided semantics, enabling semantic-driven motion alignment and generalization to unseen species. We evaluate our method on DeformingThings4D-skl, a novel quadruped motion dataset we curate. Both qualitative and quantitative evaluations demonstrate significant improvements over state-of-the-art methods, achieving superior motion naturalness and behavioral consistency across species.

This work addresses the limitations of existing motion transfer methods, which rely on predefined human skeletal structures and annotated data, thereby struggling to generalize across species. To overcome these constraints, the authors propose Motion4Motion—a training-free, cross-species motion transfer framework that eschews explicit skeleton modeling in favor of optical flow representations derived directly from video. By aligning motion features across domains during inference, the method enables high-quality motion transfer between diverse subjects—including across species—without requiring retraining or fine-tuning. This approach significantly enhances generalization capability and practical flexibility, outperforming current baselines across a range of characters and unlocking novel applications in animation production and beyond.

Existing approaches to robotic skill transfer often struggle to jointly capture semantic intent and motion dynamics, limiting their generalization, robustness, and deployment efficiency. This work proposes BooST, a two-stage framework that explicitly integrates semantics and dynamics for the first time. By leveraging a cross-modal Vector-Quantized Variational Autoencoder (VQ-VAE), BooST constructs a unified skill representation, which is then distilled—through skill abstraction and policy distillation—into a lightweight, deployable policy. Evaluated in both simulation and real-world robotic settings, the method demonstrates exceptional few-shot adaptation capabilities, strong cross-task and cross-domain transfer performance, and robustness to dynamic visual perturbations, all while maintaining computational efficiency for practical deployment.

This work addresses the challenges of high annotation costs for motion labels in real-world scenarios and performance degradation in motion prediction due to domain shift between synthetic and real data. To this end, the authors propose a dual-module transfer framework that integrates object-aware and object-assisted components, combining objectness-aware prior modeling, domain-invariant feature learning, and motion label denoising. Leveraging physics-based 4D LiDAR synthesis, they construct Motion4D—the first synthetic dataset specifically designed for motion prediction. Experimental results demonstrate that the proposed approach substantially narrows the domain gap between synthetic and real data, achieving robust and superior motion prediction performance in real-world settings.

This work addresses the challenge of sim-to-real policy transfer failures caused by unobservable dynamics—such as abrupt contacts—by introducing an inverse dynamics extraction mechanism that recovers implicit dynamical information from real-world transition data. The approach formulates dynamics transfer between simulation and reality as an unpaired domain translation task, preserving domain-specific styles while enabling effective cross-domain adaptation. By integrating physics-based simulation with real robot data, it overcomes the limitations of conventional methods that rely solely on observed history to infer latent variables. Experimental validation across humanoid, quadrupedal, and robotic arm platforms demonstrates substantially improved dynamics modeling accuracy, particularly in scenarios where observation history is insufficient or misleading. Real-world trials on the Go2 quadruped further confirm a marked enhancement in policy transfer performance.