WISE-ATTA: When to Ask for Labels in Budgeted Active Test-Time Adaptation

2026-09-29

AI Summary

This study addresses the high annotation costs and unclear timing of label requests in test-time adaptation by introducing, for the first time, a budget-constrained active test-time adaptation (ATTA) paradigm that shifts the core challenge from “what to annotate” to “when to annotate.” Methodologically, we design a budget-aware streaming scheduling algorithm that leverages lightweight online signals to dynamically determine supervision timing, alongside a drift-criterion-based sample selection mechanism to facilitate efficient model updates. Extensive experiments demonstrate that the proposed approach achieves robust performance comparable to or exceeding existing methods on benchmarks such as ImageNet-C/R/K, while requiring significantly fewer annotations.

Abstract

Active test-time adaptation (ATTA) improves robustness under distribution shift by updating a deployed model during inference while selectively querying supervision. However, most existing ATTA methods implicitly assume that supervision can be requested for every incoming test batch, which can incur substantial annotation cost over long test streams. In this work, we introduce \emph{budgeted ATTA} in which labels are available for only a fraction of test batches. This formulation shifts the central challenge from deciding \emph{what} to label within a batch to deciding \emph{when} supervision should be applied over time. To address this challenge, we propose a budget-aware approach \emph{WISE-ATTA} that allocates supervision over the test stream based on lightweight signals computed online, prioritizing periods where supervision is likely to be most useful. When a batch is selected for supervision, we further employ a drift-based sample selection criterion that targets samples exhibiting ongoing, unconverged adaptation dynamics, enabling effective updates from a single labeled example. We evaluate this approach on synthetic corruptions (ImageNet-C) and natural distribution shifts (ImageNet-R/K/A). Across settings, WISE-ATTA achieves competitive or improved performance compared to recent ATTA methods while requiring substantially fewer labels. Overall, we find that the timing of supervision is a key, yet underexplored, aspect of active test-time adaptation. Code: https://github.com/Muhammad-Huzaifaa/WISE-ATTA

Problem

• Active Test-Time Adaptation
• Budgeted Adaptation
• Distribution Shift
• Annotation Budget

Innovation

• Active Test-Time Adaptation
• Budgeted ATTA
• Supervision Timing
• Drift-based Sample Selection
• Test-Time Adaptation