Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution

2026-07-20

AI Summary

This work addresses two key limitations in existing discrete diffusion-based super-resolution methods: insufficient reconstruction of rare textures due to the long-tailed distribution of visual tokens, and spatially inconsistent artifacts caused by parallel decoding. To overcome these issues, the authors propose DiMOO-SR, a novel framework that uniquely integrates rarity-aware learning with spatially consistent decoding. Specifically, inverse frequency sampling (IFS) is employed during training to enhance the model’s focus on critical rare tokens, while spatial consistency ranking (SCR) is introduced at inference to promote structural coherence. Evaluated on multiple real-world super-resolution benchmarks, DiMOO-SR achieves high-quality image reconstruction with only a few parallel decoding steps, effectively mitigating token imbalance and structural inconsistency, thereby demonstrating the strong potential of discrete diffusion models in generative super-resolution.

Abstract

Continuous diffusion models have become the dominant paradigm for photo-realistic image Super-Resolution (SR), but they typically formulate reconstruction as continuous signal-level denoising and incorporate semantic priors through external conditioning modules. This makes it less direct to exploit the unified token-based scaling paradigm of modern multimodal models. Autoregressive models provide a more native semantic representation by modeling images as discrete visual tokens, yet their causal decoding is inefficient for high-resolution reconstruction. Discrete diffusion offers a promising middle ground by enabling non-causal, parallel prediction over visual tokens. However, directly adapting discrete diffusion to SR remains non-trivial due to two task-specific challenges: (1) the long-tailed distribution of visual tokens, which under-represents rare but perceptually critical textures; and (2) spatially inconsistent parallel decoding, which may introduce isolated artifacts. To address these issues, we propose DiMOO-SR, a rarity-aware multimodal discrete diffusion framework for photo-realistic image SR. During training, Inverse Frequency Sampling (IFS) prioritizes under-represented but information-rich tokens. During inference, Spatial Consistency Ranking (SCR) refines token confidence using local neighborhood agreement to improve structural coherence. Extensive experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps, highlighting the potential of discrete diffusion for generative image super-resolution. The code will be released upon publication.

Problem

• discrete diffusion
• image super-resolution
• long-tailed distribution
• spatial consistency
• visual tokens

Innovation

• discrete diffusion
• rarity-aware sampling
• spatial consistency
• visual tokens
• image super-resolution