Foundation-Assisted Active Learning for Object Detection Annotation

2026-07-18

AI Summary

This work addresses key challenges in remote sensing object detection, including high annotation costs, substantial localization noise during cold-start phases, and the difficulty of existing active learning methods in disentangling localization and classification uncertainties. To overcome these issues, the authors propose a foundation model–assisted active learning and semi-automatic annotation framework that fuses a reference localization source (SA-source, built upon UPN+SAM2) with a detector prediction source (OD-source) to jointly model localization consistency and classification confidence. The approach incorporates object-level features to enable diversity-aware sampling and suppress geometric noise, and introduces a dual-source bounding box switching mechanism to refine the annotation process. Experiments on DIOR, HRSC2016, DOTAv2, and FAIR1M demonstrate that the method significantly improves sample efficiency in cold-start scenarios and enhances detection performance under low annotation budgets.

Abstract

The annotation cost for remote sensing object detection is high, while existing active learning methods still face several challenges in object detection scenarios, including the coupling of localization and classification uncertainty, severe localization noise in the cold-start stage, and pseudo-diversity caused by high-recall candidate proposals. To address these issues, we propose a foundation-model-collaborative active learning and semi-automatic annotation framework for efficient construction of remote sensing object detection datasets. We build a dual-source mechanism consisting of a reference localization source (SA-source) based on UPN+SAM2 and a detector prediction source (OD-source), and further propose a Foundation-model-enhanced Dual-Source Uncertainty estimation to improve sample selection quality in the cold-start stage by jointly modeling localization consistency and classification confidence. Furthermore, we propose Object-Centric Diversity Sampling, which constructs object-level representations using DINOv2 features and SAM2 masks to improve sample coverage while suppressing pseudo-diversity. To address geometric noise in the semi-automatic annotation stage, we design Dual-Source Box Switching, which replaces noisy detector boxes with matched refined boxes from the SA-source, thereby reducing the manual burden of box refinement. Experiments on DIOR, HRSC2016, DOTAv2, and FAIR1M show that our method achieves superior or comparable results under most annotation budgets, with notably stronger cold-start sample efficiency in the low-budget regime.

Problem

• active learning
• object detection
• remote sensing
• annotation cost
• cold-start

Innovation

• Foundation Model
• Active Learning
• Object Detection
• Uncertainty Estimation
• Semi-automatic Annotation