Beyond Patches: Superpixel Token-based Transformers for Attribute-Specific Fashion Retrieval

2026-06-09

AI Summary

This work addresses the limitations of existing patch-based Transformer approaches in attribute-specific fashion retrieval, which struggle to accurately model irregular attribute regions and are susceptible to background noise. To overcome these challenges, the authors propose SuperFashion, a novel framework that introduces superpixel tokens into the Transformer architecture for the first time. Specifically, an attribute-guided attention mechanism localizes semantic regions, upon which semantically coherent superpixel tokens are generated. These tokens are then combined with modality-specific embeddings to enable adaptive cross-modal interaction and fusion. By moving beyond conventional patch-based representations, SuperFashion significantly enhances attribute localization accuracy and discriminative power. Experimental results demonstrate consistent improvements over state-of-the-art methods, with relative mAP gains of 1.84%, 9.27%, and 9.35% on the FashionAI, DARN, and DeepFashion datasets, respectively.

Abstract

Attribute-Specific Fashion Retrieval (ASFR) aims to improve fine-grained image retrieval by focusing on specific attributes. However, existing patch-based attention and Transformer methods often misalign with irregular attribute regions and are prone to background noise, limiting their ability to capture subtle, pixel-level microstructures. To tackle these challenges, we propose SuperFashion, the first ASFR framework that adopts superpixel tokens within a Transformer architecture. SuperFashion initially employs an attribute-guided attention mechanism to extract attribute-related features, which in turn guide the cropping of semantically meaningful image regions. Superpixel segmentation is then leveraged on these regions to generate compact, semantically coherent superpixel tokens. By incorporating modality-specific embeddings for both attribute and superpixel tokens, the superpixel token-based Transformer facilitates adaptive interaction and fusion, thereby enhancing attribute localization and discrimination. Extensive experiments on FashionAI, DARN, and DeepFashion demonstrate relative overall MAP improvements of 1.84%, 9.27%, and 9.35% over prior SOTA. SuperFashion offers a new solution for web-based image retrieval.

Problem

• Attribute-Specific Fashion Retrieval
• patch-based attention
• background noise
• irregular attribute regions
• pixel-level microstructures

Innovation

• superpixel token
• attribute-specific retrieval
• Transformer architecture
• attribute-guided attention
• fashion image retrieval