Per-Stroke Temporal Control for Text-to-Motion via Action Units and Action-Detection Guidance

2026-07-17

AI Summary

Current text-to-motion generation methods struggle to precisely control the timing of motion strokes—such as punches—often resulting in blurred or merged actions. This work proposes explicitly modeling each stroke via Action Units (AUs), which encode the involved body parts, action category, temporal window, and precise impact moment. To achieve this without retraining, the authors introduce a lightweight gated adapter and a dual-stream injection mechanism that incorporates frame-level detector gradients into a frozen backbone model at no additional training cost. This approach is the first to decompose motion into explicit, type- and timing-constrained conditional signals. Evaluated on the StrokeBench benchmark, it substantially improves temporal accuracy for individual strokes while maintaining or even surpassing the motion quality of existing methods, further demonstrating that keyframes can serve as an effective controllable dimension.

Abstract

Text-to-motion models are competent at the action a prompt names but unreliable at when each stroke lands: four punches alternating left and right rarely return four separable strokes. We introduce typed temporal events called Action Units (AUs) that make the individual stroke -- its body track, action class, time window, and impact timing -- an explicit conditioning signal. We ground a frozen text-to-motion backbone on the AU set through a lightweight gated adapter injecting two streams (per-stroke tokens and a per-frame phase channel), and at inference close residual timing errors with a training-free classifier gradient from a frozen frame-level detector. We measure per-stroke control on StrokeBench, whose prompts specify count, ordering, track, and core-frame placement, paired with an audited stroke corpus. AU grounding markedly raises the rate of correctly placed single strokes over the strongest prior interface, at the best motion quality among text-, interval-, and frame-level baselines. The prompted core frame emerges as a further steerable axis.

Problem

• text-to-motion
• temporal control
• action units
• stroke timing
• motion generation

Innovation

• Action Units
• Per-Stroke Temporal Control
• Text-to-Motion
• Temporal Conditioning
• Classifier Guidance