Towards More General Control of Diffusion Models Using Jeffrey Guidance

2026-06-11

AI Summary

Existing guidance methods for diffusion models struggle to explicitly represent complex target distributions, often constrained by implicit sampling rules or heuristic energy functions. This work introduces Jeffrey’s conditioning rule into diffusion guidance for the first time, proposing a Jeffrey-guided framework that explicitly updates the marginal distribution to a specified target while preserving the conditional structure, thereby overcoming the representational limitations of conventional approaches. By integrating embedded distribution matching with fairness-aware optimization constraints, the method achieves significantly lower FID scores on CIFAR-10 and FFHQ and effectively enforces attribute independence on CelebA-HQ, enhancing generation fairness.

Abstract

A key strength of diffusion models lies in their flexibility, since their outputs can be controlled at sampling time through guidance. However, beyond simple cases such as conditional sampling, the target distribution is often left implicit, defined only through a sampling rule or a heuristic energy function. To address this, we propose Jeffrey guidance, a principled framework that extends diffusion-model control to applications beyond what standard guidance can express. It leverages Jeffrey's rule of conditioning to update marginal distributions towards a prescribed target, preserving the conditional structure and minimally perturbing the joint distribution. We first demonstrate Jeffrey guidance by targeting a prescribed embedding distribution. With Inception embeddings as the target, this leads to substantial reductions in FID on both CIFAR-10 and FFHQ. We further apply Jeffrey guidance to fairness on CelebA-HQ, updating an unconditional diffusion model to enforce independence between attributes.

Problem

• diffusion models
• guidance
• target distribution
• Jeffrey's rule
• conditional sampling

Innovation

• Jeffrey guidance
• diffusion models
• distribution control
• conditional generation
• fairness