Private and Stable Test-Time Adaptation with Differential Privacy

2026-06-01

AI Summary

This work addresses the privacy risks inherent in test-time adaptation (TTA), where model updates based on historical test samples may leak sensitive information. For the first time, it integrates several prominent TTA methods—including Tent, EATA, and SAR—into a differential privacy (DP) framework. By applying per-sample gradient clipping and injecting Gaussian noise during inference, the proposed approach enables privacy-preserving adaptation with rigorous guarantees. Experiments on ImageNet-C demonstrate that the method incurs only minor accuracy degradation while effectively safeguarding privacy. Notably, under low privacy budgets, per-sample clipping not only maintains but even enhances adaptation accuracy and stability, all with manageable computational overhead.

Abstract

Test-time adaptation (TTA) can reduce error on new and different data by updating the model on these inputs during inference. However, these updates raise the issue of privacy w.r.t. the testing data, because the model parameters now depend on all past inputs. To control this privacy risk, we cast multiple popular TTA methods (Tent, EATA, SAR, DeYO, and COME) into differential privacy (DP) forms that apply per-sample gradient clipping and Gaussian noise for all updates. On ImageNet-C, our DP-TTA methods provide adequate privacy at small cost to accuracy, and in the low-privacy regime the clipping mechanism of DP can even improve the accuracy and stability of adaptation in the continual setting. These improvements to privacy and accuracy come at only modest computational overhead. These first results on private TTA raise awareness of the issue, inform the development of more private test-time updates, and identify per-sample clipping as an effective technique for improving the accuracy and stability of adaptation.

Problem

• test-time adaptation
• privacy
• differential privacy
• model updates
• testing data

Innovation

• Test-Time Adaptation
• Differential Privacy
• Gradient Clipping
• Privacy-Preserving Machine Learning
• Continual Adaptation