Adversarial Training for Pixel Diffusion

2026-09-29

AI Summary

This study addresses the deficiency of high-frequency details and insufficient distributional fidelity in images generated by pixel-space diffusion models. We present the first systematic investigation of adversarial post-training for such models. Specifically, while preserving the original diffusion objective, we apply an adversarial loss to refine predictions at non-high-noise timesteps, revealing that direct access to image statistics is critical for success. By integrating pretrained models, flow matching, and frequency-band and power-law analyses, our approach substantially recovers missing high-frequency spectral energy. Consequently, it comprehensively enhances distributional fidelity, coverage, prompt alignment, and perceptual quality.

Abstract

Pixel diffusion models generate RGB images directly, avoiding the bottleneck of an autoencoder, yet their outputs still systematically underrepresent fine-scale natural-image statistics. We show that adversarial learning provides an effective post-training correction for this deficiency. Starting from a pretrained model, we retain its original diffusion or flow-matching objective and add an adversarial loss to the predicted output at non-high-noise timesteps, leaving the model architecture and sampling procedure unchanged. To our knowledge, this is the first systematic study of adversarial post-training for pixel diffusion. Across two pixel backbones, the method jointly improves distribution fidelity, coverage, prompt alignment, and perceptual quality. We further investigate why it works. Frequency-band and power-law analyses show that the original models systematically underproduce natural-image high-frequency content, while adversarial post-training restores this missing spectral power. In contrast, perceptual loss also increases high-frequency content but sacrifices distribution fidelity and prompt alignment. Nearest-neighbor, recall, and matched no-GAN SFT controls further rule out memorization, mode dropping, and additional optimization as simple explanations. Finally, we examine the boundary of this effect. Under the tested latent diffusion configurations, the same procedure does not produce comparable joint gains and adds almost no decoded high-frequency power. These results identify direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds.

Problem

• pixel diffusion models
• adversarial training
• high-frequency content
• natural-image statistics
• fine-scale details

Innovation

• Adversarial Post-Training
• Pixel Diffusion Models
• High-Frequency Restoration
• Distribution Fidelity
• Flow Matching