Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation

2026-06-04

AI Summary

Existing Transformer-based text-to-image generation models often suffer from limited output diversity under fixed prompts due to premature convergence of the generation trajectory. This work identifies, for the first time, that the early stabilization of the zero-frequency (DC) component in intermediate features is a key cause of this issue. To address it, the authors propose DAVE—a training-free, low-overhead representation-level intervention method that selectively attenuates DC components during early generation stages to enhance diversity. Notably, DAVE requires no modification to the sampling pipeline and incurs negligible computational overhead, yet it significantly improves consistency-aware diversity while maintaining competitive image quality.

Abstract

Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text-image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this issue, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this observation, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.

Problem

• text-to-image generation
• sample diversity
• mode collapse
• representation homogeneity
• prompt-consistent variation

Innovation

• DC attenuation
• text-to-image generation
• diversity enhancement
• representation modulation
• trajectory lock-in