AsyncPatch Diffusion: spatially-flexible image generation

2026-06-05

AI Summary

This work addresses the limitation of conventional diffusion models, which employ a uniform noise schedule and thus struggle to achieve spatially adaptive denoising. The authors propose AsyncPatch Diffusion, a novel framework that assigns independent noise schedules to different image regions. They derive, for the first time, a valid evidence lower bound (ELBO) for asynchronous noise processes, enabling a single model—without fine-tuning—to perform adaptive generation, image inpainting, and input-guided synthesis. A controllable noise sampler modulates both global intensity and local variation, while input guidance enhances local consistency. The method further supports uncertainty-guided acceleration and autoregressive sampling. Evaluated on ImageNet 256×256 and LSUN, AsyncPatch Diffusion achieves generation quality on par with standard diffusion models while natively supporting high-quality image inpainting.

Abstract

Standard diffusion models corrupt an entire sample with a single shared noise level, forcing all spatial regions to follow the same denoising trajectory. We introduce AsyncPatch Diffusion, a joint-diffusion framework that assigns distinct noise levels to different input dimensions, such as image pixels, or latent tokens. We show how this asynchronous corruption defines a valid generative process while supporting a richer family of spatially heterogeneous denoising trajectories, and prove the first valid ELBO for this process. We show that a single pretrained model can perform spatially adaptive generation, where different regions are denoised on different schedules. A key challenge is training: naive independent noise-level sampling overemphasizes highly heterogeneous configurations and underrepresents homogeneous noise levels, that are crucial during sampling. We address this with a controlled noise-level sampler that regulates both the average corruption level and its spatial variability. AsyncPatch achieves generation quality comparable to conventional diffusion on ImageNet 256 and LSUN, while being natively suited for inpainting without task-specific fine-tuning. We further introduce input guidance, which uses clean or partially corrupted regions to guide the generation of unknown regions, improving local consistency and texture matching. Finally, we demonstrate adaptive generation strategies including uncertainty-guided acceleration and autoregressive sampling.

Problem

• diffusion models
• spatially-flexible generation
• asynchronous denoising
• noise-level scheduling
• image inpainting

Innovation

• AsyncPatch Diffusion
• asynchronous denoising
• spatially heterogeneous generation
• controlled noise-level sampling
• input guidance