Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation

2026-07-26

AI Summary

This work addresses the performance degradation of models during inference under continual domain shift when source data are inaccessible. To tackle this challenge, the authors propose a source-free continual test-time adaptation method based on a teacher–student framework. By dynamically adjusting the momentum of the teacher model and aligning target-domain features with class prototypes derived from the source-pretrained model, the approach effectively mitigates model drift. The method innovatively achieves fully source-free continual adaptation through an integrated design that combines exponential moving average, dynamic momentum control, class prototype estimation, and pseudo-label self-training. Extensive experiments demonstrate that the proposed approach significantly outperforms existing methods—including several state-of-the-art adaptation frameworks that rely on access to source data—across multiple benchmark datasets.

Abstract

In many real-world scenarios, encountering continual shifts in domain during inference is very common. Consequently, continual test-time adaptation (CTTA) techniques leveraging a teacher-student framework have gained prominence, allowing models to adapt continuously even after deployment. In such a framework, a weight-averaged mean teacher is used to produce pseudo-labels from test data for self-training. The mean teacher gets updated as an exponential moving average of the student parameters using a high value of momentum that is kept fixed even if different distributions of test data are encountered. To combat the resulting drift of the model, we propose a novel controlled teacher adaptation methodology that dynamically sets a proper momentum value depending on the quality of the incoming data. Additionally, we estimate class prototypes from the source pretrained model to help align the target data as they come in. Importantly, our method does not require access to source data or its statistics at any stage of the pipeline, making it truly source-free. We perform extensive experiments on benchmark datasets to demonstrate that our approach outperforms different state-of-the-art adaptation frameworks, many of which require access to source data.

Problem

• continual test-time adaptation
• domain shift
• teacher-student framework
• source-free
• model drift

Innovation

• source-free
• continual test-time adaptation
• controlled teacher adaptation
• dynamic momentum
• class prototypes