CC-Diff: Enhancing Contextual Coherence in Remote Sensing Image Synthesis

2024-12-11

arXiv.org

AI Summary

Existing remote sensing image synthesis methods primarily focus on foreground control while neglecting semantic correlations between foreground and background, resulting in scene incoherence and insufficient realism. To address this, we propose CC-Diff, a diffusion-based framework for foreground-guided sequential background generation. CC-Diff introduces a dual-role learnable query mechanism that jointly models background texture complexity and foreground-background spatial-semantic dependencies. It establishes the first foreground-conditioned background generation pipeline, enabling multi-stage serialized synthesis. Evaluated on both remote sensing (DOTA) and natural image (COCO) benchmarks, CC-Diff outperforms state-of-the-art methods: it improves detection mAP by 2.04 and 2.25 points, respectively, while significantly enhancing visual fidelity, semantic accuracy, and object localization precision.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Accurately depicting real-world landscapes in remote sensing (RS) images requires precise alignment between objects and their environment. However, most existing synthesis methods for natural images prioritize foreground control, often reducing the background to plain textures. This neglects the interaction between foreground and background, which can lead to incoherence in RS scenarios. In this paper, we introduce CC-Diff, a Diffusion Model-based approach for RS image generation with enhanced Context Coherence. To capture spatial interdependence, we propose a sequential pipeline where background generation is conditioned on synthesized foreground instances. Distinct learnable queries are also employed to model both the complex background texture and its semantic relation to the foreground. Extensive experiments demonstrate that CC-Diff outperforms state-of-the-art methods in visual fidelity, semantic accuracy, and positional precision, excelling in both RS and natural image domains. CC-Diff also shows strong trainability, improving detection accuracy by 2.04 mAP on DOTA and 2.25 mAP on the COCO benchmark.

Problem

• Enhances coherence in remote sensing image synthesis
• Addresses foreground-background correlation in image generation
• Improves realism and accuracy in synthesized images

Innovation

• Diffusion Model-based RS image generation
• Dual Re-sampler with Context Bridge
• Foreground-aware attention mechanism