Incremental Object Detection with Prompt-based Methods

2025-08-20

AI Summary

This work pioneers the application of visual prompt learning to incremental object detection (IOD), specifically addressing domain-incremental learning scenarios. To fill the gap left by existing methods—which overlook IOD— we propose a practical framework that freezes the backbone network, optimizes only lightweight visual prompts, and incorporates minimal historical data replay. Through systematic evaluation of diverse prompt architectures and ablation studies on prompt length and initialization strategies, we find that prompt-only approaches suffer from performance limitations; however, integrating an ultra-lightweight replay mechanism significantly enhances both stability and accuracy. Our approach not only demonstrates the feasibility of prompt learning in complex, structured vision tasks like object detection, but also achieves new state-of-the-art results across multiple IOD benchmarks—attaining superior generalization with minimal parameter overhead.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Visual prompt-based methods have seen growing interest in incremental learning (IL) for image classification. These approaches learn additional embedding vectors while keeping the model frozen, making them efficient to train. However, no prior work has applied such methods to incremental object detection (IOD), leaving their generalizability unclear. In this paper, we analyze three different prompt-based methods under a complex domain-incremental learning setting. We additionally provide a wide range of reference baselines for comparison. Empirically, we show that the prompt-based approaches we tested underperform in this setting. However, a strong yet practical method, combining visual prompts with replaying a small portion of previous data, achieves the best results. Together with additional experiments on prompt length and initialization, our findings offer valuable insights for advancing prompt-based IL in IOD.

Problem

• Applying prompt-based methods to incremental object detection
• Evaluating prompt approaches under domain-incremental learning settings
• Addressing performance limitations of pure prompt methods in IOD

Innovation

• Prompt-based methods for incremental object detection
• Combining visual prompts with data replay
• Analyzing prompt length and initialization impacts