CoFi-UCGen: Coarse-to-Fine Unsupervised Conditional Generation without Label Priors

2026-06-03

AI Summary

This work addresses the challenge of achieving multi-granularity semantic control in image generation under unsupervised settings, particularly when manual annotations are unavailable. The authors propose a coarse-to-fine unsupervised conditional generation framework that ensures semantic consistency between generated images and the latent space through adversarial semantic mutual learning. They construct a structured coarse-granularity latent space using binary codes and introduce a hierarchical modulation mechanism to inject conditioning signals layer-by-layer into a diffusion model, thereby enabling fine-grained attribute control. Notably, this approach is the first to simultaneously support both coarse- and fine-grained semantic manipulation without requiring labels or pretrained feature extractors, outperforming existing unsupervised methods in terms of image quality, semantic fidelity, and control precision.

Abstract

Unsupervised conditional image generation (UCGen) aims to control generation without relying on manually annotated labels, yet remains challenging due to unstructured semantic representations across granularities. To address this, we propose a novel coarse-to-fine UCGen framework (CoFi-UCGen) that explicitly disentangles global semantics from fine-grained variations, which to the best of our knowledge, sets out the first successful attempt for both coarse- and fine-grained conditional generation without any labels. More specifically, we first propose the adversarial semantic reciprocal learning theory to ensure the semantic consistency and completeness between images and latent spaces. Based on the consistency, we propose the bit-codes to learn a structured coarse-grained latent space, and further prove distinct global semantics inherent from our bit-codes while preserving independent noise sampling for generation. Building upon these bit-codes, we establish a fine-grained semantic basis and introduce a hierarchical modulation mechanism in diffusion models, by enabling layer-wise injection from coarse conditions to progressively control fine-grained attributes during generation. Extensive experiments demonstrate that without any label priors or pre-trained feature extractors, our CoFi-UCGen consistently outperforms existing UCGen methods in terms of image quality, semantic consistency, and control accuracy, verifying the effectiveness of explicit coarse-to-fine semantic decomposition for the challenging UCGen task.

Problem

• Unsupervised Conditional Generation
• Semantic Disentanglement
• Coarse-to-Fine Generation
• Label-Free Control
• Image Generation

Innovation

• unsupervised conditional generation
• coarse-to-fine disentanglement
• bit-codes
• hierarchical modulation
• diffusion models