OmniGen-AR: AutoRegressive Any-to-Image Generation

2026-06-08

AI Summary

Existing autoregressive vision generation models typically support only a single modality of conditioning, limiting their ability to meet real-world demands for image synthesis driven by multiple types of control signals. This work proposes OmniGen-AR, a unified autoregressive framework that discretizes diverse multimodal conditions—including text, spatial layouts, and visual context—into a shared token space via a common visual-text tokenizer. To prevent information leakage during training, the model introduces Decoupled Causal Attention (DCA), which separates causal dependencies between conditioning and content tokens while preserving the standard autoregressive prediction pipeline during inference. Evaluated on benchmarks such as GenEval (0.63) and VBench (80.02), OmniGen-AR achieves state-of-the-art or competitive performance, demonstrating its effectiveness in flexible, high-fidelity multimodal image generation.

Abstract

Autoregressive (AR) models have demonstrated strong potential in visual generation, offering superior performance with simple architectures and optimization objectives. However, existing methods are typically limited to single-modality conditions, e.g., text, restricting their applicability in real-world scenarios that demand image synthesis from diverse controls. In this work, we present OmniGen-AR, a unified autoregressive framework for Any-to-Image generation. By discretizing various visual conditions through a shared visual tokenizer and text prompts with a text tokenizer, OmniGen-AR supports a broad spectrum of conditional inputs within a single model, including text (text-to-image generation), spatial signals (segmentation-to-image and depth-to-image), and visual context (image editing, frame prediction, and text-to-video generation). To mitigate the risk of information leakage from condition tokens to content tokens, we introduce Disentangled Causal Attention (DCA), which separates the full-sequence causal mask into condition causal attention and content causal attention. It serves as a training-time regularizer without affecting the standard next-token prediction during inference. With this design, OmniGen-AR achieves new state-of-the-art or at least competitive results across a range of benchmark, e.g., 0.63 on GenEval and 80.02 on VBench, demonstrating its effectiveness in flexible and high-fidelity visual generation.

Problem

• Autoregressive models
• Any-to-Image generation
• Multimodal conditioning
• Visual generation
• Conditional image synthesis

Innovation

• Autoregressive Generation
• Any-to-Image
• Disentangled Causal Attention
• Unified Multimodal Conditioning
• Visual Tokenization