Embodied Active Learning under Limited Annotation and Navigation Budget for Object Detection

2026-07-17

AI Summary

This work proposes an embodied batch active learning approach for efficiently retraining object detectors in unknown environments under dual constraints of navigation time and annotation budget. By jointly optimizing robot trajectories and image sampling, the method focuses on regions where the model fails, guided by a novel spatial consistency analysis that identifies label-inconsistent samples without external supervision. This enables the agent to prioritize the collection of highly informative data. Integrated with a YOLOv5 real-time detector, the approach is validated on both the AI2-THOR simulation platform and a physical Spot robot. Experimental results demonstrate that, under identical budget constraints, the proposed method achieves significantly higher detection accuracy than multiple baselines, confirming the effectiveness of leveraging spatial inconsistency to guide active sampling.

Abstract

This paper studies how to adapt a computer vision object detector to an unknown environment under both a robot navigation time and annotation budget constraint. Our approach selects informative robot trajectories and image samples to retrain the detector, explicitly targeting its failure cases. Formally, the approach is an embodied variant of batch active learning, where at each round an agent has a limited navigation budget to collect candidate samples and a limited annotation budget for the most relevant images. We leverage spatial consistency to identify images with inconsistent labels, which are likely to provide the greatest improvement to the vision model. We evaluate the approach using different active learning objectives on large scenes from the AI2-THOR simulator and on a real-world setup using a Boston Dynamics Spot robot with the real-time object detector YOLOv5. Through comparison against several baselines, our experimental results show that spatial inconsistency helps guide the agent and select relevant images without external supervision, achieving the highest detection accuracy at the end of the adaptation process under the same budget. The open-source project can be found at https://mkabouri.github.io/embodied-active-learning-od

Problem

• Embodied Active Learning
• Object Detection
• Annotation Budget
• Navigation Budget
• Limited Supervision

Innovation

• embodied active learning
• spatial consistency
• object detection
• navigation budget
• annotation budget