Hierarchical Multi-Positive Contrastive Learning for Patent Image Retrieval

2025-06-16

AI Summary

Patent image retrieval faces challenges due to the semantic complexity of technical drawings, difficulty in domain-specific adaptation, and the absence of explicit modeling for hierarchical structural relationships. Method: This paper introduces, for the first time, the Locarno International Classification (LIC) hierarchy into contrastive learning, proposing a hierarchical multi-positive contrastive loss that enables fine-grained semantic alignment across broad categories → subcategories → specific designs. The approach integrates hierarchy-aware sampling, multi-positive instance construction, and lightweight fine-tuning of vision-language models (e.g., ViT, CLIP), specifically adapted to the DeepPatent2 dataset. Contribution/Results: Experiments demonstrate a 12.8% improvement in mean Average Precision (mAP) with only marginal parameter overhead, alongside a 35% reduction in computational cost. The method significantly enhances cross-level fine-grained matching capability and deployment efficiency.

Technology Category

Computer Vision: Image and Video RetrievalSearch and Optimization: Learning to SearchMachine Learning: Multi-instance/Multi-view Learning

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Patent images are technical drawings that convey information about a patent's innovation. Patent image retrieval systems aim to search in vast collections and retrieve the most relevant images. Despite recent advances in information retrieval, patent images still pose significant challenges due to their technical intricacies and complex semantic information, requiring efficient fine-tuning for domain adaptation. Current methods neglect patents' hierarchical relationships, such as those defined by the Locarno International Classification (LIC) system, which groups broad categories (e.g.,"furnishing") into subclasses (e.g.,"seats"and"beds") and further into specific patent designs. In this work, we introduce a hierarchical multi-positive contrastive loss that leverages the LIC's taxonomy to induce such relations in the retrieval process. Our approach assigns multiple positive pairs to each patent image within a batch, with varying similarity scores based on the hierarchical taxonomy. Our experimental analysis with various vision and multimodal models on the DeepPatent2 dataset shows that the proposed method enhances the retrieval results. Notably, our method is effective with low-parameter models, which require fewer computational resources and can be deployed on environments with limited hardware.

Problem

• Patent image retrieval faces challenges due to technical intricacies.
• Current methods ignore hierarchical relationships in patent classifications.
• Efficient fine-tuning for domain adaptation in retrieval is needed.

Innovation

• Hierarchical multi-positive contrastive loss
• Leverages LIC taxonomy for retrieval
• Effective with low-parameter models