Open-Set Object Detection By Aligning Known Class Representations

2024-01-03

IEEE Workshop/Winter Conference on Applications of Computer Vision

AI Summary

This work addresses the challenges of reliable unknown-class object identification and known/unknown class discrimination in open-set object detection (OSOD). To this end, we propose a semantic alignment clustering framework: (1) contrastive learning-driven semantic space alignment enables natural clustering of unknown classes; (2) a class decorrelation module suppresses interference from known classes to enhance unknown-class discriminability; and (3) an object-focusing module jointly predicts objectness scores and confidence, coupled with a confidence-penalty evaluation mechanism. Furthermore, we introduce a unified metric—Harmonic Mean of Precision and Recall (HMP)—that jointly quantifies known-class detection accuracy and unknown-class recall. Extensive experiments on MS-COCO and PASCAL VOC demonstrate that our method achieves state-of-the-art performance in both HMP and unknown-class recall, significantly improving the robustness and generalization capability of OSOD systems.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Open-Set Object Detection (OSOD) has emerged as a contemporary research direction to address the detection of unknown objects. Recently, few works have achieved remarkable performance in the OSOD task by employing contrastive clustering to separate unknown classes. In contrast, we propose a new semantic clustering-based approach to facilitate a meaningful alignment of clusters in semantic space and introduce a class decorrelation module to enhance inter-cluster separation. Our approach further incorporates an object focus module to predict objectness scores, which enhances the detection of unknown objects. Further, we employ i) an evaluation technique that penalizes low-confidence outputs to mitigate the risk of misclassification of the unknown objects and ii) a new metric called HMP that combines known and unknown precision using harmonic mean. Our extensive experiments demonstrate that the proposed model achieves significant improvement on the MS-COCO & PASCAL VOC dataset for the OSOD task.

Problem

• Open Set Object Detection
• Unknown Object Recognition
• Detection Accuracy

Innovation

• Semantic Clustering
• Prediction Scoring Module
• Harmonic Mean Precision (HMP)