Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editing

2026-05-31

AI Summary

This work addresses the challenge in text-driven 3D human motion editing of simultaneously preserving the stylistic characteristics of the source motion while accurately adhering to textual instructions. To this end, the authors propose a dual-axis anchored Transformer architecture that separately extracts features along the joint and temporal dimensions and integrates them through a cross-axis fusion module for joint modeling. A novel auxiliary task—joint-level motion difference prediction—is introduced, leveraging Soft-DTW distance regression to guide the model in identifying critical joints and time steps requiring modification. Integrated within a diffusion-based generative framework and evaluated on the newly curated MotionFix dataset, the method significantly enhances both semantic alignment with input text and fidelity to the original motion structure, achieving state-of-the-art performance.

Abstract

We address text-based 3D human motion editing, where the goal is to preserve the style and structure of a source motion while applying edits described in natural language. The release of the MotionFix dataset has spurred active research into training-based diffusion models that directly generate an edited motion from a source motion and a text instruction. While previous works have focused primarily on learning when an edit should occur temporally, our goal is to create a model that understands not only this temporal aspect but also which specific joints are responsible for the change. Targeting this, we propose a novel architecture and a complementary auxiliary task to aid its training. Our architecture consists of two axis-anchored transformers, which extract distinct features along the joint and time dimensions respectively, and a cross-axis fusion block that integrates these representations. We further introduce an auxiliary task that trains the joint-anchored transformer to regress the Soft-DTW distance between source and target joint rotations. This objective teaches the module to understand which joints to modify and which to preserve. Through comprehensive experiments on the MotionFix dataset, we demonstrate that our method significantly improves semantic alignment with both the text instruction and the source motion, as well as the overall fidelity of the generated motion, achieving state-of-the-art results.

Problem

• text-based motion editing
• 3D human motion
• joint-wise editing
• motion preservation
• semantic alignment

Innovation

• Cross-Axis Feature Fusion
• Joint-Wise Motion Difference Prediction
• Text-Based 3D Motion Editing
• Axis-Anchored Transformer
• Soft-DTW Regression