Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning

2026-07-23

AI Summary

This work addresses the lack of efficient computational scaling strategies in existing diffusion and flow-matching models during inference, which typically operate with fixed memory overhead and struggle to optimize generation quality. The authors propose Progressive Seed Pruning (PSP), a novel approach that, under a fixed total compute budget, initially explores multiple noise seeds in parallel and dynamically prunes low-potential trajectories based on intermediate denoising results, reserving full denoising only for the most promising ones. PSP introduces dynamic pruning into diffusion model inference for the first time, establishing a new paradigm that trades variable memory usage for enhanced output quality. Experiments demonstrate that PSP significantly outperforms baseline methods—including best-of-N sampling, importance sampling, and tree search—on both GenEval automatic evaluation and human-aligned prompt-based assessments, effectively improving reward-guided generation quality.

Abstract

Diffusion and flow-matching models dominate conditional image generation, yet inference-time scaling for these models is far less developed than for autoregressive language models. Because final quality is highly sensitive to the initial noise seed, many approaches spend extra compute on seed search or resampling under a black-box reward, but typically maintaining a constant memory footprint throughout inference. We show that relaxing this constraint enables an underexplored inference-time scaling axis: by front-loading exploration, evaluating many seeds early, and pruning aggressively, we can use a fixed compute budget more effectively. \emph{Progressive Seed Pruning} (\PSP) scores intermediate denoised estimates and progressively narrows the candidate set so that only promising trajectories are fully denoised, while keeping the total number of model evaluations fixed. Across diffusion and flow-matching backbones, \PSP \ consistently improves reward-guided selection and achieves higher GenEval scores (automated) and better human evaluation on prompt-alignment than best-of-$N$, importance-sampling, and tree-search baselines at matched compute. Project page: https://www.vision.caltech.edu/psp. Code: https://github.com/rogerioagjr/psp.

Problem

• diffusion models
• inference-time scaling
• seed pruning
• conditional image generation
• compute budget

Innovation

• Progressive Seed Pruning
• Inference-Time Scaling
• Diffusion Models
• Flow Matching
• Seed Pruning