Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentation

2025-05-19

AI Summary

Open-set recognition (OSR) faces a fundamental challenge: models struggle to learn generalizable, discriminative features robust to unseen classes. To address this, we propose GradMix—a gradient-attribution-based dynamic data augmentation method that leverages input gradient maps during training to identify and mask regions associated with learned concepts, followed by masked mixing to actively encourage the model to attend to broader, more robust discriminative features. Our key contribution is the first end-to-end differentiable integration of gradient attribution maps for dynamic sample mixing, augmented with feature-space regularization. Extensive experiments demonstrate that GradMix consistently outperforms state-of-the-art methods across three core tasks: OSR, closed-set classification, and out-of-distribution detection. Moreover, it enhances model robustness against input perturbations and improves downstream performance in self-supervised learning scenarios.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Open set recognition (OSR) is devised to address the problem of detecting novel classes during model inference. Even in recent vision models, this remains an open issue which is receiving increasing attention. Thereby, a crucial challenge is to learn features that are relevant for unseen categories from given data, for which these features might not be discriminative. To facilitate this process and"optimize to learn"more diverse features, we propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training to mask out already learned concepts. Thus GradMix encourages the model to learn a more complete set of representative features from the same data source. Extensive experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method can often outperform the state-of-the-art. GradMix can further increase model robustness to corruptions as well as downstream classification performance for self-supervised learning, indicating its benefit for model generalization.

Problem

• Detecting novel classes during model inference in open set recognition
• Learning diverse features for unseen categories from limited data
• Improving model robustness and generalization via attribution-based augmentation

Innovation

• Uses gradient-based attribution maps for masking
• Encourages learning diverse representative features
• Improves model robustness and generalization