🤖 AI Summary
Label noise fundamentally constrains object detection performance, imposing a theoretical upper bound—termed *label convergence*—on mean Average Precision (mAP), independent of model capacity.
Method: We introduce and empirically estimate label convergence on real-world, multi-source datasets (e.g., LVIS) via statistical confidence interval analysis, modeling of contradictory annotations, and standard COCO-style mAP@[0.5:0.95:0.05] evaluation.
Contribution/Results: On LVIS, label convergence is estimated at 62.63–67.52 (95% CI); state-of-the-art models already operate near this ceiling. Our analysis demonstrates that improving annotation quality yields greater performance gains than architectural innovations, providing the first quantitative evidence and methodological framework supporting data-centric AI for object detection. This work establishes label convergence as a critical diagnostic metric, shifting focus from model-centric optimization to systematic label curation as the primary lever for突破 performance bottlenecks.
📝 Abstract
Annotation errors are a challenge not only during training of machine learning models, but also during their evaluation. Label variations and inaccuracies in datasets often manifest as contradictory examples that deviate from established labeling conventions. Such inconsistencies, when significant, prevent models from achieving optimal performance on metrics such as mean Average Precision (mAP). We introduce the notion of"label convergence"to describe the highest achievable performance under the constraint of contradictory test annotations, essentially defining an upper bound on model accuracy. Recognizing that noise is an inherent characteristic of all data, our study analyzes five real-world datasets, including the LVIS dataset, to investigate the phenomenon of label convergence. We approximate that label convergence is between 62.63-67.52 mAP@[0.5:0.95:0.05] for LVIS with 95% confidence, attributing these bounds to the presence of real annotation errors. With current state-of-the-art (SOTA) models at the upper end of the label convergence interval for the well-studied LVIS dataset, we conclude that model capacity is sufficient to solve current object detection problems. Therefore, future efforts should focus on three key aspects: (1) updating the problem specification and adjusting evaluation practices to account for unavoidable label noise, (2) creating cleaner data, especially test data, and (3) including multi-annotated data to investigate annotation variation and make these issues visible from the outset.