Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

2026-07-27

AI Summary

This work addresses the instability in knowledge transfer observed in existing policy distillation methods under classifier-free guidance (CFG), which stems from the asymmetric treatment of negative branches—a phenomenon termed the Negative Branch Asymmetry (NBA) problem. The study elucidates this failure mechanism and introduces a novel distillation objective, termed Positive-direction Matching (PDM), that separately constrains the forward prediction and the conditional guidance direction. By integrating branch-level velocity matching with a trajectory query mechanism, the proposed approach explicitly compensates for errors in both positive and negative branches under CFG. Evaluated on dense-to-sparse video control tasks, the method substantially enhances distillation stability, reduces sensitivity to inference guidance scales, and improves overall knowledge transfer efficacy.

Abstract

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model's native CFG schema retains privileged information in the teacher's negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive--Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

Problem

• On-Policy Distillation
• Classifier-Free Guidance
• Negative Branch Asymmetry
• Diffusion Models
• Branch Error Dynamics

Innovation

• On-Policy Distillation
• Classifier-Free Guidance
• Negative Branch Asymmetry
• Positive–Direction Matching
• Diffusion Models