Revisiting Autoregressive Models for Generative Image Classification

2026-03-19

AI Summary

This work addresses a key limitation of conventional autoregressive generative image classifiers, which rely on a fixed token ordering that introduces inductive biases detrimental to image understanding and constrains classification performance. To overcome this, the paper proposes the first framework for generative image classification based on arbitrary-order autoregressive modeling. By generating predictions across diverse token orderings and marginalizing over these outcomes through averaging, the method yields more robust and comprehensive classification signals. This approach effectively circumvents the representational constraints imposed by fixed sequences, achieving significant performance gains over diffusion-based classifiers—up to 25× greater efficiency—while matching the accuracy of state-of-the-art self-supervised discriminative models on multiple image classification benchmarks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Class-conditional generative models have emerged as accurate and robust classifiers, with diffusion models demonstrating clear advantages over other visual generative paradigms, including autoregressive (AR) models. In this work, we revisit visual AR-based generative classifiers and identify an important limitation of prior approaches: their reliance on a fixed token order, which imposes a restrictive inductive bias for image understanding. We observe that single-order predictions rely more on partial discriminative cues, while averaging over multiple token orders provides a more comprehensive signal. Based on this insight, we leverage recent any-order AR models to estimate order-marginalized predictions, unlocking the high classification potential of AR models. Our approach consistently outperforms diffusion-based classifiers across diverse image classification benchmarks, while being up to 25x more efficient. Compared to state-of-the-art self-supervised discriminative models, our method delivers competitive classification performance - a notable achievement for generative classifiers.

Problem

• autoregressive models
• generative image classification
• token order
• inductive bias
• class-conditional generative models

Innovation

• autoregressive models
• any-order generation
• order marginalization
• generative classification
• token order