B-DENSE: Branching For Dense Ensemble Network Learning

2026-02-17

AI Summary

Diffusion model distillation often suffers from structural information loss and discretization errors due to sparse supervision, degrading generation quality. To address this, this work proposes B-DENSE, a novel framework that introduces a multi-branch trajectory alignment mechanism. By employing a K-branch channel architecture, B-DENSE enables the student network to concurrently model multiple intermediate steps of the teacher model, thereby establishing dense intermediate supervision. This design allows the student to learn the full teacher trajectory from the early stages of training, significantly enhancing both generation quality and trajectory fidelity. Experimental results demonstrate that B-DENSE consistently outperforms existing distillation baselines on image generation tasks and effectively mitigates the performance degradation caused by sparse supervision.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Inspired by non-equilibrium thermodynamics, diffusion models have achieved state-of-the-art performance in generative modeling. However, their iterative sampling nature results in high inference latency. While recent distillation techniques accelerate sampling, they discard intermediate trajectory steps. This sparse supervision leads to a loss of structural information and introduces significant discretization errors. To mitigate this, we propose B-DENSE, a novel framework that leverages multi-branch trajectory alignment. We modify the student architecture to output $K$-fold expanded channels, where each subset corresponds to a specific branch representing a discrete intermediate step in the teacher's trajectory. By training these branches to simultaneously map to the entire sequence of the teacher's target timesteps, we enforce dense intermediate trajectory alignment. Consequently, the student model learns to navigate the solution space from the earliest stages of training, demonstrating superior image generation quality compared to baseline distillation frameworks.

Problem

• diffusion models
• inference latency
• distillation
• trajectory alignment
• discretization errors

Innovation

• diffusion distillation
• trajectory alignment
• multi-branch architecture
• dense supervision
• generative modeling