Ranking vs. Assignment: The Metric Mismatch in Multi-View Object Association

2026-06-01

AI Summary

This work addresses a fundamental misalignment between prevailing pairwise ranking metrics—such as Average Precision (AP) and FPR-95—and the actual objective of one-to-one assignment in multi-view object association tasks. The study theoretically demonstrates, for the first time, that even with perfectly correct assignments, these ranking metrics can yield suboptimal scores, and conversely, optimal ranking does not guarantee correct assignment. To expose this disconnect, the authors propose Sinkhorn normalization as a lightweight post-processing step and validate the phenomenon through controlled stress tests and sensitivity analyses. Experiments reveal that modest tuning of post-processing parameters can substantially improve AP and FPR-95 without corresponding gains in assignment-level metrics like Accuracy (ACC) or IPAA, thereby highlighting critical limitations in current evaluation protocols and offering new insights for future metric design.

Abstract

Multi-view object association is an important computer vision problem that underlies many multi-camera perception tasks. While this task is naturally formulated as a constrained one-to-one matching problem, recent works heavily rely on pairwise ranking metrics like AP and FPR-95 for model evaluation. We highlight a fundamental mismatch between these metrics and the actual assignment objective. Theoretically, we show that AP and FPR-95 can be imperfect even when the assignment is already correct, and that Sinkhorn-based normalization can make them perfect. Conversely, optimal pairwise ranking can still lead to incorrect assignments. We validate this mismatch in practice by using our Sinkhorn-based normalization as a controlled post-processing stress test. We show that optimizing just a few post-processing parameters significantly boosts AP and FPR-95 without corresponding improvements in assignment-level metrics such as ACC and IPAA.

Problem

• multi-view object association
• metric mismatch
• assignment problem
• pairwise ranking
• evaluation metrics

Innovation

• multi-view object association
• metric mismatch
• Sinkhorn normalization
• assignment problem
• pairwise ranking metrics