COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relations

2026-06-03

AI Summary

This work addresses the challenge in compositional image retrieval where visually similar yet attribute-irrelevant samples interfere with multimodal feature fusion and similarity modeling. To mitigate this issue, the authors propose a cross-modal unified representation framework grounded in attribute prototypes, comprising three core components: adaptive semantic disentanglement, Compositional Unified Prototypes (CUP), and dual relational modeling. This approach effectively disentangles semantics and jointly captures attribute-level pairwise and neighborhood relationships. As the first systematic study to analyze the impact of such interference, the paper introduces an attribute-prototype-based similarity metric that significantly enhances retrieval accuracy and robustness, achieving state-of-the-art performance across three benchmark datasets.

Abstract

Composed Image Retrieval (CIR) represents a challenging retrieval task that targets locating specific images through multimodal inputs. Despite recent progress in CIR techniques, prior approaches often overlook cases where images appear visually alike yet differ in attributes, potentially undermining both multimodal feature fusion and similarity modeling. To mitigate this limitation, we design a unified representation of cross-modal features based on attribute prototypes. Nevertheless, the task is far from straightforward, owing to three core issues: (1) entanglement in attribute-level semantics, (2) inconsistency across modalities, and (3) supervised signal missing. To tackle the above obstacles, we introduce a COMposed image retrieval network guided By attrIbute-based NEighbor Relations (COMBINER). Specifically, we first design an Adaptive Semantic Disentanglement module, which is capable of disentangling attribute features based on multimodal primitive features. Secondly, we propose a Unified Prototype-based Composition module, which can construct cross-modal unified prototypes (CUP) and facilitate multimodal feature composition. Finally, we introduce a Dual Relations Modeling module, which can mine pairwise and neighbor relations based on attribute similarity. Compared to traditional neighbor relations modeling CIR methods, COMBINER represents the first study addressing the phenomenon of visually similar but attribute-unrelated samples. It achieves a more accurate understanding of the semantic relations among samples by employing an attribute prototype-based similarity metric. Comprehensive experiments conducted on three benchmark datasets confirm the effectiveness of our proposed COMBINER. The implementation of our method will be accessed at https://github.com/Lee-zixu/COMBINER

Problem

• Composed Image Retrieval
• attribute-based similarity
• visually similar but attribute-unrelated samples
• multimodal feature fusion
• semantic relations

Innovation

• Composed Image Retrieval
• Attribute Prototypes
• Semantic Disentanglement
• Cross-modal Fusion
• Neighbor Relations Modeling