VISTA: Video-Injected Stylized Text-to-Animation

2026-09-20

AI Summary

VISTA通过融合文本提示的内容和参考视频的风格，生成3D人体动作，使用双通道自动编码器和扩散模型解决非配对数据下的动画生成问题。

Abstract

We present VISTA, a two-stage framework for generating stylized 3D human motion by fusing structural content from text prompts with expressive style from reference videos, without requiring jointly paired (text, video, stylized motion) triplets. A Dual-channel Autoencoder first maps motion sequences and video clips into a shared latent manifold. A masked autoregressive diffusion backbone then operates within this manifold, injecting video-derived style through a dedicated late-fusion Dual-AdaLN pathway while preserving text-conditioned content structure. A cross-batch unpaired training protocol with latent cycle consistency enables joint learning across separate semantically rich and stylistically diverse datasets. As a proof-of-concept for controllable animation synthesis, we validate VISTA on rendered motion-capture references: it achieves the highest style recognition accuracy among video-conditioned methods while preserving competitive content alignment, and its decomposed 3-way classifier-free guidance provides independent, user-controllable calibration of the content--style balance at inference time.

Problem

• stylized 3D human motion
• text prompts
• reference videos
• unpaired data
• style and content fusion

Innovation

• Dual-channel Autoencoder
• masked autoregressive diffusion
• Dual-AdaLN
• latent cycle consistency
• classifier-free guidance