Coarse-to-Fine Compositional Diffusion for Long-Horizon Planning

2026-05-30

AI Summary

Existing diffusion models struggle to maintain global structural coherence in long-horizon tasks due to their reliance on stitching together locally generated plans. This work proposes a coarse-to-fine compositional diffusion approach that decouples global structure modeling from local detail generation. It first constructs a global skeleton through coarse-grained sampling to ensure long-term consistency, then refines fine-grained details by leveraging a pretrained short-horizon diffusion prior initialized from an intermediate noise level. Evaluated on robotic planning, panoramic image synthesis, and long video generation, the method significantly outperforms existing approaches—simultaneously improving both global coherence and local fidelity while reducing denoiser invocation counts by 2–8×.

Abstract

Diffusion models provide strong priors for generating structured data, but many tasks require outputs beyond the scale on which these models are typically trained. Compositional generation addresses this by composing overlapping local plans from a pretrained short-horizon prior into a long-horizon output. However, standard composition primarily enforces agreement between neighboring local plans, yielding local consistency without directly specifying the global structure of the full composition. As a result, locally compatible plans may still form an implausible route, task sequence, or temporal evolution. Existing methods improve global coherence by repeatedly propagating local consistency signals or by adding inference-time optimization, but these procedures become expensive as the number or dimensionality of local plans increases. We propose Coarse-to-Fine Compositional Diffusion (CoFi), an inference-time sampler that separates global structure formation from local detail refinement. CoFi first aligns local denoised estimates around a shared coarse structure, producing a global scaffold that captures the long-range task-level arrangement. It then diffuses this scaffold to an intermediate noise level and denoises it with the same pretrained local prior, restoring local fine structure while preserving the scaffold-induced global coherence. Across long-horizon robotic planning, panoramic image generation, and long video generation, CoFi not only improves both global coherence and local sample quality over prior compositional baselines, but also requires 2-8x fewer denoiser evaluations.

Problem

• long-horizon planning
• compositional generation
• global coherence
• diffusion models
• structured data generation

Innovation

• Coarse-to-Fine
• Compositional Diffusion
• Long-Horizon Planning
• Global Coherence
• Diffusion Models