DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformers

2026-08-03

AI Summary

This work addresses the lack of systematic understanding and effective metrics for inter-block representation diversity in existing Diffusion Transformers (DiTs). We introduce, for the first time, a Weighted Diversity Score (WDS) that quantitatively reveals a strong negative correlation (r = −0.869) between representation diversity and generation quality. Building upon this insight, we propose DiverseDiT++, a framework that explicitly encourages diverse representation learning through long-range residual connections and a dedicated diversity loss function. Our method achieves significant improvements in both generation performance and convergence speed on ImageNet at 256×256 and 512×512 resolutions, while seamlessly integrating with DiT backbones of varying scales.

Abstract

Recent advances in Diffusion Transformers (DiTs) have enabled remarkable progress in visual synthesis, benefiting from their superior scalability. To facilitate DiTs' capability of capturing meaningful internal representations, recent works such as REPA incorporate external pretrained encoders for representation alignment. However, the underlying mechanisms governing representation learning within DiTs remain poorly understood in the community. To this end, this paper first presents a systematic analysis of the representation dynamics of DiTs via quantifying the diversity of block-wise representations. Specifically, we introduce a novel metric, termed the Weighted Diversity Score (WDS), to measure the representational discrepancies across different blocks. Through extensive investigations on the evolution and influence of internal representations under various settings, we reveal that representation diversity across blocks is a critical factor for effective representation learning in DiTs. More importantly, WDS exhibits a strong correlation with synthesis quality across diverse settings, model scales, and training stages (Pearson's $r=-0.869$ with $\log(\text{FID})$), suggesting its potential as an indicator to reflect model performance and a principled guide for model optimization. Based on this key finding, we propose DiverseDiT++, a novel framework that explicitly promotes diverse representation learning. Concretely, our method incorporates long residual connections to diversify input representations across blocks and a representation diversity loss to encourage blocks to learn distinct features. Extensive experiments on ImageNet $256\times256$ and $512\times512$ demonstrate that our DiverseDiT++ yields consistent performance gains and convergence acceleration when applied to different backbones with various sizes,...

Problem

• Diffusion Transformers
• representation diversity
• representation learning
• visual synthesis
• internal representations

Innovation

• Representation Diversity
• Diffusion Transformers
• Weighted Diversity Score
• DiverseDiT++
• Residual Connections