Fixed-point neural samplers on discrete spaces

2026-10-01

AI Summary

This study addresses the challenges of mode collapse, lack of convergence guarantees, and dependence on specific reference processes in sampling from discrete distributions by proposing a Discrete Gibbs Iterative Neural Sampler. By integrating masked diffusion models with fixed-point iteration algorithms, this method enables efficient and scalable training of neural generative models while overcoming reliance on particular diffusion processes. The framework provides theoretical convergence guarantees, effectively mitigates mode collapse, and supports both inter-distribution transport and amortized sampling. Experimental results demonstrate that the proposed approach scales favorably to high-dimensional systems and is successfully applied to the accurate estimation of alloy phase diagrams.

Abstract

Sampling from discrete, unnormalized distributions without access to data is a challenging problem. Neural samplers offer a promising approach by training generative models from density evaluations directly. Despite recent progress, existing discrete neural samplers are prone to mode collapse, come without convergence guarantees when trained via fixed-point iterations, and are often tied to a specific reference process such as masked or uniform diffusion. In this work, we introduce Discrete Gibbs Iterative Neural Sampler, a fixed-point neural sampler that addresses these limitations, enabling efficient, scalable learning, substantially reducing mode collapse in practice. Our framework builds on masked diffusion and also extends to transport between pairs of distributions. We demonstrate that the resulting method scales effectively to high-dimensional systems, supports amortized sampling across different conditions, and enables accurate estimation of alloy phase diagrams.

Problem

• discrete sampling
• unnormalized distributions
• neural samplers
• mode collapse
• fixed-point iterations

Innovation

• discrete neural sampler
• fixed-point iteration
• masked diffusion
• mode collapse
• amortized sampling