🤖 AI Summary
Existing attention-based sparse image matching models exhibit significant performance variations across different local features, yet the individual contributions of detectors and descriptors remain unclear. This work systematically investigates this issue and reveals that the choice of detector has a far greater impact on matching performance than that of the descriptor. Building on this insight, we propose a general, detector-agnostic zero-shot matching strategy: by fine-tuning a Transformer-based matcher with keypoints aggregated from multiple pre-existing detectors, our approach eliminates the need for retraining on any specific detector. Experiments demonstrate that, in zero-shot settings, our method achieves matching accuracy on novel detectors that matches or even surpasses that of models explicitly trained for those detectors, confirming its effectiveness and strong generalization capability.
📝 Abstract
We revisit the problem of training attention-based sparse image matching models for various local features. We first identify one critical design choice that has been previously overlooked, which significantly impacts the performance of the LightGlue model. We then investigate the role of detectors and descriptors within the transformer-based matching framework, finding that detectors, rather than descriptors, are often the primary cause for performance difference. Finally, we propose a novel approach to fine-tune existing image matching models using keypoints from a diverse set of detectors, resulting in a universal, detector-agnostic model. When deployed as a zero-shot matcher for novel detectors, the resulting model achieves or exceeds the accuracy of models specifically trained for those features. Our findings offer valuable insights for the deployment of transformer-based matching models and the future design of local features.