DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding

2026-08-05

AI Summary

Existing block-wise speculative decoding models suffer from a significant drop in acceptance length under high-entropy target distributions due to their assumption of conditional independence among token positions within a draft block. To address this limitation, this work proposes Dependency-aware Block Speculative Decoding (DBLAST), which explicitly models inter-positional dependencies within draft blocks for the first time. DBLAST captures cross-positional token correlations through a low-rank latent variable mixture and introduces a training strategy optimized to maximize verified sequence length. Experiments on Qwen3-4B and Qwen3-8B demonstrate that DBLAST substantially outperforms independent block sampling across diverse benchmarks—including GSM8K, MT-Bench, HumanEval, and creative writing tasks—with consistent gains in both acceptance length and speculation efficiency, particularly in high-entropy decoding scenarios.

Abstract

Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to verify them. While recent block and diffusion-style drafters can predict several positions in a single pass, their training and sampling procedures are typically optimized for greedy decoding or assume that positions in the draft block are conditionally independent. This assumption becomes brittle in non-greedy speculative decoding, where the target distribution is deliberately stochastic and multiple continuations become plausible. We study this mismatch for block diffusion drafters and show that the accepted draft length degrades as the entropy of the target sampling distribution increases. We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective that directly targets the expected verified length. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, MT-Bench, HumanEval, and creative-writing benchmarks show that our approach, namely DBLast, consistently improves accepted length over independent block sampling, especially in higher-entropy decoding regimes.

Problem

• speculative decoding
• block drafting
• stochastic decoding
• conditional independence
• draft acceptance

Innovation

• speculative decoding
• dependent block drafting
• stochastic decoding
• low-rank latent mixture
• acceptance-oriented training