Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy

2026-06-05

AI Summary

Traditional human image animation methods rely on explicit pose estimation, which is highly susceptible to occlusions and complex poses, often degrading generation quality. This work proposes DirectAnimator, an end-to-end framework for driving video-based animation that eliminates the need for intermediate pose representations. The approach introduces a Driving Cue Triplet that jointly encodes pose, facial expression, and spatial location cues, and incorporates a novel CueFusion DiT module to enable effective multi-cue collaboration. Furthermore, a Same2X training strategy is devised to enhance cross-identity generalization while preserving source identity fidelity. Experimental results demonstrate that DirectAnimator consistently outperforms existing methods in terms of visual quality, robustness, and computational efficiency.

Abstract

Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.

Problem

• human image animation
• pose estimation
• occlusion
• complex poses
• identity preservation

Innovation

• Direct animation
• Driving Cue Triplet
• CueFusion DiT
• Same2X training
• Pose-free animation