SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation

2026-03-02

AI Summary

This work addresses the high inference latency of autoregressive image generation models, which stems from sequential token-by-token decoding. Existing training-free acceleration methods verify tokens independently, neglecting their strong co-occurrence patterns and thereby causing contextual inconsistencies that limit efficiency. To overcome this, we propose the first training-free, phrase-level speculative verification framework. By mining co-occurrence statistics of visual tokens from the training corpus, our method constructs semantically coherent visual phrases and jointly verifies multiple tokens within each decoding window. Acceptance of an entire phrase is determined by an aggregated likelihood ratio, departing from the conventional assumption of independent single-token verification. This approach effectively captures local co-occurrence regularities, reducing function evaluations by up to 30% in text-to-image generation while maintaining generation quality and significantly accelerating decoding.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify tokens independently, overlooking the strong co-occurrence patterns between adjacent visual tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window. To construct such phrase units, we analyze token co-occurrence statistics from the training corpus and group frequently co-occurring tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.

Problem

• autoregressive image generation
• inference latency
• token co-occurrence
• decoding efficiency
• contextual inconsistency

Innovation

• speculative decoding
• phrase-level verification
• autoregressive image generation
• token co-occurrence
• inference acceleration