Adaptive Estimation and Learning under Temporal Distribution Shift

2025-05-21

AI Summary

This paper addresses high-precision point estimation of the ground-truth value at the final time step under temporal distribution shift. For noisy, nonstationary, time-varying ground-truth sequences, we propose an adaptive estimation framework combining wavelet soft-thresholding with sparse modeling. We establish, for the first time, a theoretical connection between nonstationarity strength and sparsity in the wavelet domain, enabling derivation of the optimal point estimation error bound without prior knowledge of shift intensity. We further extend the theory to binary classification, deriving a sparse-aware excess risk upper bound and designing a computationally efficient novel training objective. Our core contributions are: (1) a unifying theoretical bridge linking nonstationarity and sparsity; (2) a tight error bound for final-time ground-truth estimation; and (3) a generalizable risk analysis and optimization paradigm applicable to supervised learning.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we study the problem of estimation and learning under temporal distribution shift. Consider an observation sequence of length $n$, which is a noisy realization of a time-varying groundtruth sequence. Our focus is to develop methods to estimate the groundtruth at the final time-step while providing sharp point-wise estimation error rates. We show that, without prior knowledge on the level of temporal shift, a wavelet soft-thresholding estimator provides an optimal estimation error bound for the groundtruth. Our proposed estimation method generalizes existing researches Mazzetto and Upfal (2023) by establishing a connection between the sequence's non-stationarity level and the sparsity in the wavelet-transformed domain. Our theoretical findings are validated by numerical experiments. Additionally, we applied the estimator to derive sparsity-aware excess risk bounds for binary classification under distribution shift and to develop computationally efficient training objectives. As a final contribution, we draw parallels between our results and the classical signal processing problem of total-variation denoising (Mammen and van de Geer,1997; Tibshirani, 2014), uncovering novel optimal algorithms for such task.

Problem

• Estimating time-varying groundtruth under temporal distribution shift
• Developing optimal wavelet-based estimation without shift knowledge
• Connecting non-stationarity to wavelet sparsity for error bounds

Innovation

• Wavelet soft-thresholding for optimal error bounds
• Links non-stationarity to wavelet-domain sparsity
• Sparsity-aware risk bounds for classification