DynamicEarth: How Far are We from Open-Vocabulary Change Detection?

2025-01-22

AI Summary

Existing open-category change detection methods in Earth observation are constrained by predefined semantic categories, limiting their ability to identify previously unseen semantic changes. Method: We introduce Open-Vocabulary Change Detection (OVCD), a novel task that eliminates reliance on category priors inherent in conventional supervised or unsupervised paradigms. Our approach proposes a training-free dual-framework—Mask-then-Clip-and-Identify (M-C-I) and Image-then-Mask-and-Clip (I-M-C)—integrating multimodal foundation models (SAM, DINOv2, Grounding-DINO, and SAM2) for zero-shot change localization and semantic classification via vision-language alignment. Contribution/Results: We release DynamicEarth, the first open-source OVCD codebase and evaluation suite. Extensive experiments across five benchmarks demonstrate substantial improvements over state-of-the-art methods, validating strong generalization and robustness to unseen classes and domain shifts.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Unsupervised & Self-Supervised Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Monitoring Earth's evolving land covers requires methods capable of detecting changes across a wide range of categories and contexts. Existing change detection methods are hindered by their dependency on predefined classes, reducing their effectiveness in open-world applications. To address this issue, we introduce open-vocabulary change detection (OVCD), a novel task that bridges vision and language to detect changes across any category. Considering the lack of high-quality data and annotation, we propose two training-free frameworks, M-C-I and I-M-C, which leverage and integrate off-the-shelf foundation models for the OVCD task. The insight behind the M-C-I framework is to discover all potential changes and then classify these changes, while the insight of I-M-C framework is to identify all targets of interest and then determine whether their states have changed. Based on these two frameworks, we instantiate to obtain several methods, e.g., SAM-DINOv2-SegEarth-OV, Grounding-DINO-SAM2-DINO, etc. Extensive evaluations on 5 benchmark datasets demonstrate the superior generalization and robustness of our OVCD methods over existing supervised and unsupervised methods. To support continued exploration, we release DynamicEarth, a dedicated codebase designed to advance research and application of OVCD. https://likyoo.github.io/DynamicEarth

Problem

• Change Detection
• Real-world Complexity
• Unseen Vocabulary Variation

Innovation

• Open Vocabulary Change Detection (OVCD)
• M-C-I and I-M-C Frameworks
• DynamicEarth Project