Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections

2026-07-25

AI Summary

This work addresses the challenge of verifying covariate balance in covariate shift adaptation by proposing a sequentially valid, anytime-stoppable validation framework. Built upon time-uniform confidence sequences, the method dynamically monitors covariate balance for a pre-specified function class within a prescribed tolerance band and terminates as soon as all target moments fall within this band, thereby certifying balance. Its key contribution lies in providing, for the first time, a locally and absolutely valid certification of balance for any adjustment strategy, supporting data-dependent stopping times while rigorously controlling the probability of erroneous balance confirmation. Integrated with KL-divergence-based drift diagnostics and detection of admissible adjustment regions, experiments demonstrate the method’s superior performance in error rate control, locality with respect to function classes, effectiveness in drift diagnosis, and guaranteed coverage in conformal prediction.

Abstract

Many covariate-shift adaptation methods construct a correction $w(x)$, but users must still determine whether the corrected distribution is sufficiently balanced for the target stream. We study anytime-valid confirmation of prespecified corrections from sequential unlabeled target inputs. Our primary contribution is a procedure for confirming covariate balance. For a prespecified class of balancing functions and tolerances, time-uniform confidence sequences permit continuous monitoring and data-dependent stopping once all plausible target moments lie within their tolerance bands. If the correction is out of tolerance for at least one function, the probability of ever incorrectly confirming balance is at most the prescribed level. Upon stopping, the procedure yields a certificate local to the chosen functions and tolerances, yet providing an absolute downstream-adequacy statement that ordinary shift diagnostics generally do not. With finite source data, contracted bands preserve this guarantee while accounting for uncertainty in weighted source moments, whereas expanded bands support only compatibility diagnostics. As complementary information, we study a source-calibrated likelihood-ratio e-process whose KL-drift identity characterizes correction directions relative to the source. Under the source-reference distribution, the probability of ever crossing its evidence threshold is controlled, but crossing does not confirm balance. We also give an exponential-tilt test for departures beyond an acceptable correction region and deploy balance-confirmed corrections in weighted conformal prediction. Experiments illustrate false-confirmation control, locality to the balancing-function class, KL-drift diagnostics, acceptable-region monitoring, finite-source effects, and downstream conformal coverage under covariate shift.

Problem

• covariate balance
• anytime-valid inference
• sequential monitoring
• covariate shift
• confidence sequences

Innovation

• anytime-valid inference
• covariate balance confirmation
• confidence sequences
• e-process
• weighted conformal prediction