QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation

2026-10-07

AI Summary

This study addresses the imbalance between two-dimensional spatial binding and one-dimensional sequential flexibility in conventional visual tokenization, as well as the token redundancy caused by fixed grids. To overcome these limitations, this work proposes QuadTok, a hierarchical quadtree-based visual tokenizer that dynamically allocates resolution to configure representational capacity on demand. This approach effectively balances spatial structure with sequential causality and integrates a GPT-style autoregressive model to enable zero-shot spatially controllable generation. Experimental results demonstrate that QuadTok reduces token usage by approximately 10% while achieving a gFID of 2.08 on ImageNet reconstruction, and attains 9% token compression on the COCO dataset. These findings indicate that the proposed framework significantly enhances both the efficiency and controllability of image generation.

Abstract

We introduce QuadTok, a novel framework for visual tokenization and autoregressive image generation. Compared to traditional approaches using 2D grids or 1D token sequences, we propose a hierarchical quadtree structure, bridging the gap between 2D spatial binding and 1D sequence-level flexibility. The QuadTok tokenizer dynamically allocates representational capacity to visually intricate areas while leaving homogeneous regions at a coarse resolution. Compared with a fixed 256-token grid, our ImageNet-trained tokenizer saves approximately 10% of tokens on ImageNet and 9% when transferred zero-shot to the COCO dataset, while maintaining comparable reconstruction fidelity. Furthermore, the natural causality introduced by the tree structure seamlessly enables autoregressive image generation. Conditioned on a quadtree topology supplied before generation, our 947M GPT-style generative model achieves a 2.08 gFID on the ImageNet $256 \times 256$ benchmark. Additionally, leveraging the strong spatial correlation preserved by the quadtree structure, the QuadTok generator enables zero-shot spatially controlled image generation capabilities. Code: https://github.com/myc634/QuadTok.

Problem

• visual tokenization
• autoregressive image generation
• quadtree structure
• dynamic token allocation
• spatially controlled generation

Innovation

• Quadtree Visual Tokenizer
• Autoregressive Image Generation
• Dynamic Token Allocation
• Zero-shot Spatial Control
• Hierarchical Structure