Deterministic Discrete Denoising

2025-09-25

AI Summary

To address the inefficiency and instability in sampling caused by stochastic denoising in discrete diffusion models, this paper proposes the first deterministic denoising framework that requires neither model retraining nor continuous embeddings. Methodologically, it constructs a deterministic reverse transition process based on a Markov chain, integrating an enhanced herding algorithm and weak chaotic dynamics to enable deterministic trajectory evolution over discrete state spaces. The core contribution is the systematic introduction of deterministic reverse processes into discrete diffusion modeling—eliminating the inherent randomness and redundant iterations of conventional sampling. Experiments demonstrate significant improvements: up to 3.2× faster sampling and enhanced sample quality (FID reduced by 18.7%, BLEU increased by 2.4%) on both text and image generation tasks. Performance matches that of continuous diffusion models, establishing a novel paradigm for discrete generative modeling.

Technology Category

Computer Vision: Diffusion Models for VisionSearch and Optimization: Mixed Discrete/Continuous SearchNatural Language Processing: Generation

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We propose a deterministic denoising algorithm for discrete-state diffusion models based on Markov chains. The generative reverse process is derandomized by introducing a variant of the herding algorithm with weakly chaotic dynamics, which induces deterministic discrete state transitions. Our approach is a direct replacement for the stochastic denoising process, requiring neither retraining nor continuous state embeddings. We demonstrate consistent improvements in both efficiency and sample quality on text and image generation tasks. Thus, this simple derandomization approach is expected to enhance the significance of discrete diffusion in generative modeling. Furthermore, our results reveal that deterministic reverse processes, well established in continuous diffusion, can also be effective in discrete state spaces.

Problem

• Proposes deterministic denoising for discrete diffusion models
• Replaces stochastic process without retraining or embeddings
• Aims to improve efficiency and quality in generation tasks

Innovation

• Deterministic discrete denoising algorithm using Markov chains
• Herding algorithm variant with weakly chaotic dynamics
• Direct replacement for stochastic process without retraining