Test-Time Generalized Category Discovery

2026-09-27

AI Summary

This study addresses the challenge of unifying known-class classification and unknown-class discovery under test-time distribution shifts by introducing, for the first time, the Test-Time Generalized Category Discovery (TT-GCD) scenario and proposing the PACT framework. This method leverages vision-language models to perform prototype estimation and zero-shot prediction realignment, adaptively adjusting visual features through prototype assignment during a purely unsupervised testing phase to unify domain adaptation with novel category discovery. Experimental results demonstrate that PACT surpasses existing state-of-the-art methods across multiple benchmarks, effectively bridging the gap between test-time adaptation and open-world discovery.

Abstract

Test-Time Adaptation (TTA) and Generalized Category Discovery (GCD) are traditionally treated as disjoint problems: the former adapts models to domain shift assuming all test classes are known, while the latter discovers novel categories assuming labeled training data for known classes. However, real-world deployment rarely fits either setting. Motivated by this gap, we introduce Test-Time Generalized Category Discovery (TT-GCD), a unified and more realistic scenario where a vision-language model must adapt to distribution shifts, classify known categories using only textual supervision, and discover novel categories, all during test time and without access to labeled data. To address this challenging scenario, we propose PACT (Prototype Assignment for Category discovery at Test time), a fully unsupervised framework that casts known-class recognition and novel-class discovery via prototype assignment. PACT first re-aligns shifted visual features with the text-derived class representations of the VLM using confident zero-shot predictions. Known and novel categories are then both represented by prototypes in the visual embedding space, estimated from the unlabeled test stream, and each test image is assigned to the category whose prototype is most similar to its visual feature. Extensive experiments across corruption and domain-shift benchmarks demonstrate that PACT outperforms adapted state-of-the-art TTA and GCD methods, effectively bridging the gap between adaptation and discovery.

Problem

• Test-Time Adaptation
• Generalized Category Discovery
• Vision-Language Model
• Novel Category Discovery
• Distribution Shift

Innovation

• Test-Time Adaptation
• Generalized Category Discovery
• Vision-Language Model
• Prototype Assignment
• Unsupervised Learning