AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding

2026-06-06

AI Summary

This work addresses the inference inefficiency in diffusion language models caused by strict inter-block dependencies in chunked semi-autoregressive decoding. The authors propose AsyncLane, a training-free asynchronous decoding scheduler that, for the first time, decouples the refinement and continuation stages in diffusion language models. Upon detecting reliable delimiters or stable semantic prefixes, AsyncLane dynamically splits decoding into parallelizable refinement and continuation paths, managing dependencies and output ordering through a path tree. Combined with optimizations—including shared-prefix batching, look-ahead draft reuse, cascaded termination, and compact cache flushing—it efficiently supports multi-path asynchronous execution under bidirectional attention. Evaluated on mathematical reasoning and code generation tasks, AsyncLane achieves up to 2.95× and 3.04× throughput improvements on LLaDA and Dream models, respectively, without compromising generation quality.

Abstract

Block-wise semi-autoregressive decoding is the standard inference paradigm for diffusion large language models (DLMs), but it imposes a strict dependency between blocks: the next block cannot begin until the current block is fully decoded or its denoising budget is exhausted. We observe that once a block exposes a reliable delimiter boundary or stable semantic prefix, continuation generation need not wait for every residual token to be resolved. We propose AsyncLane, a training-free decoding scheduler that decouples refinement from advancement. AsyncLane forks a generate lane at observed delimiter boundaries into a refine lane and a continuation generate lane: the prefix remains editable, while the continuation advances before prefix refinement finishes. The resulting lane tree records decoding dependencies and output order, while execution proceeds over the active lane set. To make this asynchronous schedule efficient under bidirectional attention, AsyncLane combines shared-prefix lane batching, lookahead draft reuse, cascading termination, and compact cache refresh with refresh-logit reuse, preventing model-call cost from scaling directly with the number of lanes. AsyncLane is a drop-in replacement for block-wise DLM samplers and requires no retraining. Experiments on mathematical reasoning and code generation show that AsyncLane consistently improves throughput while maintaining competitive quality. Across LLaDA and Dream backbones, AsyncLane achieves the highest TPS in all evaluated benchmark-length settings; relative to the fastest competing baseline, it reaches peak speedups of 2.95x on LLaDA and 3.04x on Dream, with especially large gains under longer generation budgets.

Problem

• diffusion language models
• block-wise decoding
• decoding dependency
• generation throughput
• semi-autoregressive decoding

Innovation

• AsyncLane
• diffusion language models
• semi-autoregressive decoding
• decoupled refinement
• asynchronous decoding