One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold

2026-07-19

AI Summary

This work investigates how the spatial geometric structure of low-uncertainty positions influences the dependency cost among selected locations under a confidence-based single-step parallel decoding scheme in masked diffusion models. To this end, the authors propose a locally dependent non-negative Gaussian random field model incorporating distance-dependent Gaussian correlation structures, enabling factorized decoding by selecting the K positions with the lowest scores. The study establishes, for the first time, a rigorous stochastic geometric relationship between the geometric configuration of low-score positions and their total correlation, revealing a phase transition behavior at the square-root scale: below this scale, the total correlation vanishes in probability, whereas it maintains a non-zero expected lower bound at or above it. Theoretical analysis is corroborated by synthetic experiments demonstrating accurate finite-size predictions.

Abstract

Motivated by confidence-guided parallel unmasking in masked discrete diffusion, we study a single selection step in a stylized Gaussian random-field model. A locally dependent nonnegative score field represents position wise uncertainty, and the scheduler selects the K positions with the smallest scores. Dependence among the selected positions is measured through a distance-dependent Gaussian correlation model. This separation provides a tractable framework for quantifying how the geometry of low-score locations affects the dependence cost of factorized parallel decoding. We establish two complementary results. In a conservative sub-square-root regime, the conditional Gaussian total correlation of the selected block vanishes in probability. At the square-root scale, it remains non-negligible with positive asymptotic probability and admits a strictly positive expectation lower bound. Synthetic experiments support the predicted finite-size behavior. These results provide a rigorous stochastic-geometry baseline for understanding how budget size, score dependence, and spatial correlation jointly shape one-step confidence-based selection in masked discrete diffusion.

Problem

• masked diffusion
• Gaussian random field
• total correlation
• parallel decoding
• score dependence

Innovation

• masked diffusion
• Gaussian random field
• total correlation
• parallel unmasking
• stochastic geometry