Learning Sampling Parameters for Diffusion Models

2026-07-26

AI Summary

This work addresses the limitation of static sampling parameters—such as prompts, classifier-free guidance scales, and noise schedules—in diffusion models, which fail to adapt to varying prompts and denoising stages. The authors propose LeSAMP, a novel framework that formulates sampling parameter scheduling as a reinforcement learning problem. LeSAMP leverages large language models to generate prompt-conditioned, timestep-adaptive parameter policies, optimized jointly via human preference models and a vision-language model acting as a judge (VLM-as-a-judge). Evaluated on Flux.1 [dev] and Stable Diffusion 3.5, LeSAMP significantly outperforms baseline methods, achieving a human preference win rate of 68.12%, a VLM-assessed score of 73.37%, and a user-study win rate of 59.46%, thereby establishing a new paradigm for dynamic optimization of diffusion sampling.

Abstract

Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, negative prompts, classifier-free guidance scales, and noise schedules. These parameters are typically manually chosen once and then held fixed across prompts and denoising timesteps, even though different prompts and stages of generation can benefit from different parameter values. We introduce LeSAMP, a framework for learning prompt-conditioned, timestep-varying sampling parameters. We formulate parameter selection as a reinforcement learning problem: Given a user prompt, a large language model is trained to emit schedules for the chosen sampling parameters. We optimize our model using rewards from human preference models and VLM-as-a-judge. We evaluate our model on Flux.1 [dev] and Stable Diffusion 3.5, and find that compared to baselines, LeSAMP has a win rate of up to 68.12% using human preference scores and 73.37% using VLM-as-a-judge. These gains are validated in a user study where we achieve win rates of up to 59.46% over previous baselines. Our results suggest that learned sampling-parameter policies provide a complementary approach to existing post-training methods for improving diffusion model outputs.

Problem

• diffusion models
• sampling parameters
• text-to-image generation
• parameter optimization
• timestep-varying

Innovation

• diffusion models
• sampling parameters
• reinforcement learning
• prompt-conditioned scheduling
• VLM-as-a-judge