ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification

2026-06-08

AI Summary

This work addresses the limited generalization capability of existing models to novel categories and unseen domains in open-vocabulary, open-domain object detection. To this end, the authors propose ExDet, a lightweight framework that leverages the DeltaSpace property of vision-language models to generate category- and domain-aware visual prototypes via text-guided extrapolation. Without requiring any retraining of the detector, ExDet introduces a Detector-Compatible Rectification mechanism to calibrate classification head outputs and incorporates an ExRPN proposal recalibration strategy to refine region proposal scores. Notably, ExDet achieves the first data-free and training-free enhancement of cross-category and cross-domain generalization, establishing state-of-the-art performance across multiple benchmarks, including OD-LVIS, OV-LVIS, Objects365, and MSOSB.

Abstract

Open-domain open-vocabulary detection (ODOVD) requires detectors to generalize to both novel categories and unseen domains, making it more challenging than open-vocabulary detection. Existing methods typically train open-vocabulary detectors together with domain generalization modules from scratch, leading to high training cost. we propose ExDet, a lightweight category-domain collaborative generalization framework for ODOVD that enhances the cross-category and cross-domain generalization of existing detectors. ExDet consists of Text-Guided Extrapolation (TGE), a lightweight Detector-Compatible Rectification (DCR) module, and ExRPN. Specifically, TGE exploits the DeltaSpace property of vision-language models (VLMs) to infer category- and domain-aware proxy visual prototypes from text. DCR is learned from the TGE-generated prototypes in a detector training-free and real-data-free manner, and is inserted after the classification head at inference to rectify representations toward a detector-compatible source-domain visual distribution, thereby enhancing classification for targets from novel categories and unseen domains. ExRPN recalibrates proposal scores by combining semantic similarity with RPN confidence, improving recall for novel and domain-shifted objects while providing better support for subsequent classification and DCR. ExDet achieves SOTA performance on OD-LVIS, OV-LVIS, Objects365, and MSOSB.

Problem

• open-domain
• open-vocabulary detection
• domain generalization
• novel categories
• unseen domains

Innovation

• open-domain detection
• open-vocabulary detection
• cross-modal extrapolation
• detector-compatible rectification
• vision-language models