Destruction is a General Strategy to Learn Generation; Diffusion's Strength is to Take it Seriously; Exploration is the Future

2026-05-28

AI Summary

This work addresses the limitations of generative models in data-scarce settings and the incompatibility of conventional reinforcement learning exploration mechanisms with diffusion models. It proposes a unified “destruction-as-learning” perspective, reframing the diffusion process as a flexible paradigm of information masking and reconstruction. By formalizing the diffusion mechanism through a novel probabilistic graphical model, the approach transcends handcrafted masking strategies and enhances model expressivity. Building on this foundation, the paper advocates for and initiates the development of exploration strategies natively grounded in the diffusion process, offering a new theoretical framework that bridges generative modeling and reinforcement learning, and thereby opening promising directions for data-efficient generative learning and diffusion-native exploration.

Abstract

I present diffusion models as part of a family of machine learning techniques that withhold information from a model's input and train it to guess the withheld information. I argue that diffusion's destroying approach to withholding is more flexible than typical hand-crafted information withholding techniques, providing a rich training playground that could be advantageous in some settings, notably data-scarce ones. I then address subtle issues that may arise when porting reinforcement learning techniques to the diffusion context, and wonder how such exploration problems could be addressed in more diffusion-native ways. I do not have definitive answers, but I do point my fingers in directions I deem interesting. A tutorial follows this thesis, expanding on the destroy-then-generate perspective. A novel kind of probabilistic graphical models is introduced to facilitate the tutorial's exposition.

Problem

• diffusion models
• information withholding
• reinforcement learning
• exploration
• data-scarce

Innovation

• diffusion models
• information withholding
• destroy-then-generate
• probabilistic graphical models
• exploration