Pooling Representation Autoencoders for Efficient Diffusion

2026-10-06

AI Summary

This study addresses the prohibitive computational overhead of generative modeling with representation autoencoders caused by dense token grids. To this end, we propose PoolDINO, a framework that introduces a learnable affine pooling operator to merge adjacent tokens, achieving efficient compression by exploiting local feature correlations. Furthermore, PoolDINO jointly trains an RGB decoder with an internally guided diffusion model, eliminating the need for an independent feature autoencoder while preserving the standard two-stage pipeline and significantly simplifying the architecture. Experiments on ImageNet demonstrate that PoolDINO achieves 4× token compression without compromising generation quality, yielding a 3.7× to 9.0× improvement in sampling throughput. These results indicate that the proposed method effectively balances generative efficiency with visual fidelity.

Abstract

Representation Autoencoders (RAEs) generate images from pre-trained visual fea- tures, but their dense token grids make generative modeling expensive. Motivated by local feature correlations, we introduce PoolDINO, a learned affine pooling operator that merges neighboring tokens. Training the pooling operator jointly with the RGB decoder preserves the standard two-stage RAE procedure without a separate feature auto-encoder. On ImageNet-256, 4x token compression retains comparable generation quality under internal guidance, while 16x compression trades some quality for greater efficiency. At a fixed budget of 100 sampling steps, latent-sampling throughput increases by 3.7x and 9.0x, respectively, relative to the unpooled baseline. Classification and dense prediction evaluations show that comparable guided generation quality can coexist with weaker performance on other tasks.

Problem

• Representation Autoencoders
• Diffusion Models
• Token Compression
• Computational Efficiency
• Image Generation

Innovation

• Representation Autoencoder
• Affine Pooling
• Token Compression
• Diffusion Models
• Throughput Efficiency