Acoustic Progress Propagation for Long-Horizon Speculative Decoding in ASR

2026-09-27

AI Summary

This study addresses the saturation of accepted draft length with increasing steps in long-horizon speculative decoding. To overcome this limitation, we propose a progress-aware speculative drafter that introduces recursive propagation of acoustic progress states, feeding them back into the audio cross-attention mechanism to transcend conventional alignment-aware constraints. Furthermore, the drafter and a progress predictor are jointly trained, integrated with a variable-horizon strategy to optimize the generation process. Evaluated across five ASR benchmarks, our method achieves lossless acceleration, where the accepted draft length grows consistently with the lookahead horizon, yielding substantial speedups over baseline approaches.

Abstract

Speculative decoding accelerates autoregressive automatic speech recognition (ASR), but the acceptance length of alignment-aware drafters can saturate as the draft horizon increases. We propose a progress-aware speculative drafter that recurrently propagates an acoustic progress state across draft steps and feeds it back into audio cross-attention to guide token generation. We jointly train the drafter and progress predictor over variable draft horizons. On five ASR test sets, our method achieves lossless, macro-averaged end-to-end speedups of 1.657x and 1.227x over target-only autoregressive decoding with Qwen3-ASR-0.6B and Qwen3-ASR-1.7B, respectively. Relative to AnchorDraft, our method improves the macro-averaged speedup by 34.3% and 9.0%, respectively. Horizon sweeps show continued growth in acceptance length beyond the baselines'saturation. Code is available at https://github.com/yuanyuanjia71-spec/ProgDraft.

Problem

• Speculative Decoding
• Automatic Speech Recognition
• Acceptance Length Saturation
• Long-Horizon Drafting

Innovation

• Speculative Decoding
• Acoustic Progress Propagation
• Automatic Speech Recognition
• Progress-aware Drafter
• Cross-attention