Long-Tailed Visual Recognition via Permutation-Invariant Head-to-Tail Feature Fusion

2025-05-31

AI Summary

Long-tailed class distribution severely degrades deep models’ recognition performance on tail classes, primarily due to representation space distortion and classifier bias induced by sparse tail-class features. To address this, we propose the Permutation-Invariant Head-to-Tail Feature fusion framework (PIF-H2TF), which enables unbiased semantic transfer from head to tail classes via permutation-symmetric modeling—jointly optimizing both feature representation space and decision boundaries. Our approach innovatively integrates feature reweighting, semantic distillation, and classifier calibration into a plug-and-play module compatible with mainstream backbones and long-tailed learning paradigms. Extensive experiments on standard benchmarks—including ImageNet-LT and iNaturalist—demonstrate significant improvements in tail-class accuracy, achieving new state-of-the-art overall performance without incurring additional inference overhead.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The imbalanced distribution of long-tailed data presents a significant challenge for deep learning models, causing them to prioritize head classes while neglecting tail classes. Two key factors contributing to low recognition accuracy are the deformed representation space and a biased classifier, stemming from insufficient semantic information in tail classes. To address these issues, we propose permutation-invariant and head-to-tail feature fusion (PI-H2T), a highly adaptable method. PI-H2T enhances the representation space through permutation-invariant representation fusion (PIF), yielding more clustered features and automatic class margins. Additionally, it adjusts the biased classifier by transferring semantic information from head to tail classes via head-to-tail fusion (H2TF), improving tail class diversity. Theoretical analysis and experiments show that PI-H2T optimizes both the representation space and decision boundaries. Its plug-and-play design ensures seamless integration into existing methods, providing a straightforward path to further performance improvements. Extensive experiments on long-tailed benchmarks confirm the effectiveness of PI-H2T.

Problem

• Addresses imbalanced data distribution in long-tailed recognition
• Improves tail class accuracy via feature fusion
• Optimizes representation space and decision boundaries

Innovation

• Permutation-invariant feature fusion enhances representation
• Head-to-tail fusion transfers semantic information
• Plug-and-play design integrates seamlessly