HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts

2024-08-08

arXiv.org

AI Summary

This paper addresses a novel challenge in generalized category discovery (GCD): the presence of unlabeled data drawn from multiple unknown domains—distinct from the labeled source domain—thereby inducing domain shift. To tackle this, we propose the first domain-shift-aware GCD framework. Methodologically, we formally model statistical independence between semantic and domain features, and design a HiLo dual-stream network to achieve explicit feature disentanglement. We further introduce domain-aware augmentation and curriculum learning to enhance robustness against domain shifts. Additionally, we construct the first domain-shift-oriented GCD benchmark, incorporating DomainNet and a fine-grained corrosion dataset with cross-domain annotations. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods on multi-domain GCD tasks, achieving substantial gains in both unknown-category recognition accuracy and domain generalization capability.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Application Domains: Humanities & Computational Social Science

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Generalized Category Discovery (GCD) is a challenging task in which, given a partially labelled dataset, models must categorize all unlabelled instances, regardless of whether they come from labelled categories or from new ones. In this paper, we challenge a remaining assumption in this task: that all images share the same domain. Specifically, we introduce a new task and method to handle GCD when the unlabelled data also contains images from different domains to the labelled set. Our proposed `HiLo' networks extract High-level semantic and Low-level domain features, before minimizing the mutual information between the representations. Our intuition is that the clusterings based on domain information and semantic information should be independent. We further extend our method with a specialized domain augmentation tailored for the GCD task, as well as a curriculum learning approach. Finally, we construct a benchmark from corrupted fine-grained datasets as well as a large-scale evaluation on DomainNet with real-world domain shifts, reimplementing a number of GCD baselines in this setting. We demonstrate that HiLo outperforms SoTA category discovery models by a large margin on all evaluations.

Problem

• Handles Generalized Category Discovery with domain shifts
• Separates semantic and domain features in unlabelled data
• Outperforms state-of-the-art models in category discovery

Innovation

• HiLo networks separate semantic and domain features
• Domain augmentation tailored for GCD task
• Curriculum learning approach enhances model performance