Compositional Attribute Imbalance in Vision Datasets

2025-06-17

AI Summary

This paper addresses the long-tailed distribution of both individual and compositional attributes in visual datasets. We formally define “compositional attribute imbalance” for the first time and propose a CLIP-based approach to construct an attribute dictionary and quantify attribute rarity. Building upon this, we design a rarity-driven adaptive resampling strategy and synergistically integrate advanced augmentation techniques—including CutMix, FMix, and SaliencyMix—to jointly optimize attribute distribution modeling and data augmentation. Extensive experiments across multiple benchmark datasets demonstrate that our method significantly mitigates attribute bias, enhances model robustness and fairness in recognizing long-tailed categories, and consistently improves both classification accuracy and cross-domain generalization performance.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Visual attribute imbalance is a common yet underexplored issue in image classification, significantly impacting model performance and generalization. In this work, we first define the first-level and second-level attributes of images and then introduce a CLIP-based framework to construct a visual attribute dictionary, enabling automatic evaluation of image attributes. By systematically analyzing both single-attribute imbalance and compositional attribute imbalance, we reveal how the rarity of attributes affects model performance. To tackle these challenges, we propose adjusting the sampling probability of samples based on the rarity of their compositional attributes. This strategy is further integrated with various data augmentation techniques (such as CutMix, Fmix, and SaliencyMix) to enhance the model's ability to represent rare attributes. Extensive experiments on benchmark datasets demonstrate that our method effectively mitigates attribute imbalance, thereby improving the robustness and fairness of deep neural networks. Our research highlights the importance of modeling visual attribute distributions and provides a scalable solution for long-tail image classification tasks.

Problem

• Addressing visual attribute imbalance in image classification
• Analyzing single and compositional attribute imbalance effects
• Proposing sampling adjustment and augmentation for rare attributes

Innovation

• CLIP-based framework for attribute dictionary
• Adjust sampling by compositional attribute rarity
• Integrate data augmentation for rare attributes