Beyond Resolution: Object-to-Image Ratio Mismatch in Instance Retrieval

2026-10-08

AI Summary

This study addresses the severe performance degradation in visual instance retrieval caused by object-to-image (O2I) scale mismatch, wherein the target object's relative proportion differs substantially between queries and gallery images. For the first time, this work quantifies the impact of O2I mismatch and reveals its asymmetric failure characteristics. To mitigate this issue without modifying the indexing structure, a query-side augmentation strategy is proposed that integrates multi-scale architectural analysis, OWLv2-based crop re-ranking, and LoRA fine-tuning to achieve optimization with zero training overhead. Evaluated on the ILIAS 100M benchmark, the proposed approach significantly improves mAP@1000 from 29.2 to 42.0, establishing new state-of-the-art performance and offering an efficient paradigm for large-scale instance retrieval.

Abstract

Visual instance retrieval often fails when the same object appears at different apparent sizes in the query and gallery. We show that the dominant cause is usually not resolution loss but object-to-image (O2I) ratio mismatch: the object occupies different fractions of the two images. On a controlled benchmark of 3,021 Objaverse objects rendered at five camera distances, more than 80% of the cross-distance degradation is attributable to O2I mismatch rather than resolution for 9 of 12 pretrained backbones; multi-scale architectures cut the resolution-only effect to single digits yet remain equally susceptible. The failure is also asymmetric: tight queries retrieve more reliably against wide gallery images than the reverse. Guided by this analysis, query-side scale augmentation and an OWLv2 crop reranker reach state of the art on ILIAS 100M (29.2 mAP@1000 before reranking, 42.0 after) without training or modifying the precomputed gallery index, and a LoRA fine-tune matches the query-side gains at a single forward pass, showing that O2I robustness is learnable.

Problem

• Instance Retrieval
• Object-to-Image Ratio Mismatch
• Scale Variation
• Cross-distance Degradation

Innovation

• Instance Retrieval
• Object-to-Image Ratio Mismatch
• Scale Augmentation
• Crop Reranker
• LoRA Fine-tuning