GestureFAR: Streaming Co-Speech Gesture Generation with Flow Autoregression

2026-09-18

AI Summary

为解决在线生成自然同步手势的问题，提出GestureFAR框架，利用流自回归模型处理连续动作潜在变量，提高实时性和手势质量。

Abstract

Generating natural co-speech gestures from streaming speech is essential for embodied conversational agents, where motion must be produced while a user is still speaking. Recent streaming gesture systems make online generation possible by autoregressing over discrete motion tokens, but this design compresses high-dimensional continuous motion into finite codebooks and can limit the realism and diversity of generated gestures. To preserve both causality and continuous expressiveness, we propose \textbf{GestureFAR}, a flow-autoregressive framework for streaming co-speech gesture generation. First, GestureFAR autoregresses over causal continuous motion latents, using a transformer to model streaming audio-motion context and a per-token flow-matching head to sample the next latent from a continuous distribution. Second, we introduce a head-only flow distillation strategy that freezes the causal backbone and distills the multi-step per-token flow head into a single network evaluation using consistency and distribution-matching objectives. This keeps the model token-causal while removing the main latency bottleneck for live interaction. Experiments on BEAT2 show that GestureFAR significantly improves the quality--latency trade-off among streaming-capable methods, preserving strong gesture quality while enabling real-time token-causal generation. Project Page: https://andypinxinliu.github.io/GestureFAR

Problem

• co-speech gestures
• streaming speech
• autoregression
• continuous motion
• realism and diversity

Innovation

• Flow Autoregression
• Streaming Co-Speech Gesture Generation
• Causal Continuous Motion Latents
• Head-Only Flow Distillation