Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectives

2026-09-28

AI Summary

This study addresses the exponential computational explosion caused by enumerative completion in sequence-level guidance for discrete diffusion models by proposing the COFFEE framework. This method pioneers the integration of joint conditioning, completion-weighted guidance, and optimization constraints into pretrained generation. By decoupling sequence dependencies from the objective function and employing a neuro-symbolic strategy that combines a denoiser carrier model with compiled finite-state machines to handle unresolved token distributions, it achieves plug-and-play global preference guidance without retraining. Experiments demonstrate that the framework delivers strong controllability across symbolic, linguistic, and biological benchmarks, effectively balancing generation quality and diversity. These results validate the potential of neuro-symbolic approaches for controllable generation.

Abstract

Discrete diffusion models generate sequences by iteratively resolving multiple tokens in parallel, offering a flexible alternative to left-to-right generation. However, guiding this process with a sequence-level objective is difficult because the value of one unresolved token depends on the other tokens with which it can form a high-reward sequence. Enumerating all such completions makes the whole guidance computation grow exponentially with the number of unresolved positions. We introduce COFFEE, a plug-and-play framework that avoids this enumeration by separating sequence dependence from the objective. At each diffusion step, a target-free carrier absorbs the marginal token distributions predicted by the denoiser to construct a joint model over the unresolved tokens, while a compiled finite-state model records how their combinations affect the sequence-level preference. Pairing their states allows COFFEE to transfer global preferences to unresolved positions and sample a clean reconstruction without retraining the diffusion model. The same framework supports explicit hard constraints and learned soft objectives. We evaluate COFFEE across multiple symbolic, language, and biological benchmarks, where it achieves strong control results with task-dependent quality and diversity trade-offs. By making objectives available to inference rather than only evaluation, COFFEE brings joint conditioning, completion-weighted guidance, and optimization-based constraints into pretrained neural generation, showing the potential of neural-symbolic methods in diffusion guidance.

Problem

• Discrete diffusion models
• Sequence-level objective
• Guided generation
• Exponential complexity
• Token dependence

Innovation

• Discrete Diffusion
• Finite-State Model
• Plug-and-Play Guidance
• Sequence-Level Objective
• Neural-Symbolic Methods