DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer

2026-07-20

AI Summary

Existing pixel-level diffusion Transformers compress image patches into single tokens, struggling to simultaneously preserve global structure and local details. To address this limitation, this work proposes the first dual-branch pixel-level diffusion architecture: a compact backbone branch efficiently captures global semantics, while a high-capacity pixel branch retains fine-grained textures through sub-patch grouping. The two branches interact via cross-attention mechanisms, enabling explicit decoupling of global reasoning and local modeling. This design enhances both structural consistency and textural richness in generated images while achieving a superior trade-off between generation quality and computational efficiency, outperforming current pixel-level diffusion Transformers.

Abstract

Diffusion Transformers achieve strong image generation performance, but most operate in compressed latent spaces. Pixel-space diffusion avoids this information loss, yet existing approaches map each raw image patch to a single token, forcing one representation to handle both global communication and fine-grained details. We address this issue by proposing a new architecture, \textbf{DuSPiT}, a \textbf{Du}al-branch \textbf{S}ub\textbf{P}atch \textbf{Pi}xel \textbf{T}ransformer. This model separates global structural reasoning from local appearance modeling. DuSPiT uses a compact base branch for efficient global reasoning and a parallel, high-capacity pixel branch, organized into subpatch groups, to preserve detailed appearance, with the two branches interacting through cross-attention. Our results show that DuSPiT generates images with richer details and stronger fine-grained structures, while also achieving a better quality--efficiency trade-off than prior pixel-space diffusion transformers.

Problem

• pixel-space diffusion
• image generation
• token representation
• global structure
• fine-grained details

Innovation

• Dual-branch architecture
• Pixel-space diffusion
• Sub-patch grouping
• Cross-attention interaction
• Fine-grained image generation