cross-cohort calibration

Designs, implements, and evaluates lightweight adaptation and calibration procedures that transfer a pretrained model to a new cohort by adjusting only the final decision parameters (for example re‑fitting a linear head, temperature scaling, or small adapter modules) using few labeled examples; and analyzes how these procedures affect predictive performance, probability calibration, and decision thresholds across source and target cohorts.

cross-cohortcalibration

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
0.53
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$200K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the unreliability of selecting and evaluating few-shot adaptation strategies under clinical distribution shifts by proposing the Adapter and Automator architectures. Methodologically, it defines an expanded adaptation space combined with reliability rules to automatically search for optimal strategy combinations. Furthermore, evidence-based weighted fusion and reliability screening mechanisms are introduced to achieve efficient few-shot adaptation. The approach integrates techniques from large model pre-training, few-shot learning, and AutoML. Experimental results demonstrate that the proposed method attains state-of-the-art performance on critical care datasets using only minimal patient data, providing a robust and reliable solution for transfer learning in clinical scenarios.

clinical distribution shiftsfew-shot adaptationpretrained clinical models

This study addresses prediction performance degradation caused by covariate shift by proposing an adaptive importance-weighted model averaging method. The approach constructs a family of estimators through exponentiated density ratios and treats the degree of weighting correction as a source of uncertainty. By optimizing convex combination weights in a data-driven manner, it effectively balances bias and variance to achieve asymptotically optimal prediction on the target domain. A key innovation lies in integrating importance weighting into the model averaging framework while establishing theoretical optimality. Empirical evaluations on both simulated and real-world datasets demonstrate that the proposed method achieves competitive predictive performance compared to existing approaches.

Bias-variance trade-offCovariate shiftDistributional mismatch

Conventional covariate adjustment in randomized controlled trials (RCTs) often suffers from suboptimal efficiency due to inflexible, pre-specified modeling assumptions. Method: We propose a prespecified yet data-adaptive machine learning framework for RCT analysis. It introduces the first adaptive targeted maximum likelihood estimation (TMLE) strategy that integrates sample splitting with cross-validated variance minimization—enabling data-driven model selection while strictly adhering to the pre-specified statistical analysis plan (SAP). Rigorous validation employs adaptive pre-specification, plasmode simulation, and parametric simulation. Contribution/Results: Applied to primary endpoint analyses of eight published clinical trials (2022–2024), our method significantly improves precision in marginal treatment effect estimation. It supports real-time, global remote unblinding and robust implementation, establishing a new paradigm for RCT analysis that is prespecifiable, reproducible, and statistically efficient.

Develops data-adaptive covariate adjustment to estimate marginal treatment effectsImplements targeted machine learning with adaptive pre-specification for efficiencyOptimizes precision in randomized trial analysis using machine learning

Evaluating and Utilizing Surrogate Outcomes in Covariate-Adjusted Response-Adaptive Designs

Aug 05, 2024
WZ
Wenxin Zhang
🏛️ University of California, Berkeley | Fred Hutchinson Cancer Center

To address the challenge of balancing treatment effect heterogeneity learning and decision timeliness in adaptive clinical trials, this paper proposes a covariate-adjusted response-adaptive design that accelerates randomization probability updates using surrogate outcomes. Methodologically, it introduces the first formal framework quantifying the dual benefits—accelerated learning and bias reduction—of surrogate outcomes in sequential adaptive trials; develops an Online-Superlearner–driven mechanism for dynamic surrogate selection; and establishes a model-agnostic, targeted minimum loss–based estimation (TMLE) inference method tailored to adaptive trial data. Extensive simulations—including scenarios calibrated to real trial data—demonstrate that the design significantly improves the expected outcome for newly enrolled participants while preserving asymptotic normality of estimators. Collectively, it provides a generalizable toolkit for evaluation, selection, and inference in adaptive clinical trials.

Evaluating multiple candidate adaptive designs in clinical trialsQuantifying unobserved benefits and costs of alternative designsSelecting optimal surrogate-guided designs for treatment effect detection

Latest Papers

What's happening recently
View more

This study addresses the failure of conformal prediction coverage guarantees caused by post-deployment data distribution shifts. It proposes Exponential Tilting Reweighting Alignment (ExTRA), a framework for modeling distribution shifts that systematically compares two adaptation strategies: weight calibration and prediction tilting. The analysis reveals that while prediction tilting can reduce set sizes under specific conditions, it may compromise coverage validity. Experiments demonstrate that ExTRA decreases prediction set lengths by 30% in synthetic regression tasks; however, it induces significant coverage degradation in classification or information-deficient settings and yields no consistent benefits on real-world data. Determining the precise applicability conditions for this approach remains an open challenge.

Conformal PredictionCoverage GuaranteeDistribution Shift

This work addresses the challenge of verifying covariate balance in covariate shift adaptation by proposing a sequentially valid, anytime-stoppable validation framework. Built upon time-uniform confidence sequences, the method dynamically monitors covariate balance for a pre-specified function class within a prescribed tolerance band and terminates as soon as all target moments fall within this band, thereby certifying balance. Its key contribution lies in providing, for the first time, a locally and absolutely valid certification of balance for any adjustment strategy, supporting data-dependent stopping times while rigorously controlling the probability of erroneous balance confirmation. Integrated with KL-divergence-based drift diagnostics and detection of admissible adjustment regions, experiments demonstrate the method’s superior performance in error rate control, locality with respect to function classes, effectiveness in drift diagnosis, and guaranteed coverage in conformal prediction.

anytime-valid inferenceconfidence sequencescovariate balance

This study addresses a key challenge in randomized controlled trials: how to effectively leverage covariate adjustment to improve the precision of average treatment effect estimation while satisfying regulatory requirements and ensuring statistical validity. The authors propose a prespecified, transparent, and reproducible covariate adjustment framework that, for the first time, integrates data-adaptive methods and machine learning into a regulatory-compliant analytical pipeline. By combining model-misspecification-robust estimation with semiparametric efficiency theory, the approach consistently outperforms unadjusted analyses without compromising causal interpretability or statistical validity. It substantially enhances estimation precision, increases statistical power, and yields narrower confidence intervals.

covariate adjustmentdata-adaptive methodsrandomized trials

This work addresses the challenge of enabling predictive systems to dynamically adapt their behavior based on contextual information for personalized inference. To this end, it proposes a unified framework that maps context into adaptation parameters for prediction and, for the first time, establishes a mathematical equivalence between explicit parameter adaptation and implicit expert routing under kernel ridge regression. The framework theoretically unifies diverse methodologies—including varying-coefficient models, local regression, prompt engineering, retrieval-augmented approaches, and mixture-of-experts—under fixed features and squared loss. Key contributions include deriving a general formulation for context-adaptive inference, proposing practical design principles and evaluation metrics such as adaptation efficiency and routing stability, and highlighting critical open problems concerning identifiability and robustness under distributional shifts.

context-adaptive inferencedistribution shiftfoundation models

This work addresses Bayesian optimal experimental design under computationally expensive models with limited design evaluations. It proposes an adaptive sequential elimination algorithm that significantly reduces the variance and computational cost of nested Monte Carlo estimators by reusing parameter samples, employing common random numbers, and applying Rao–Blackwellization. A bootstrap-based probabilistic comparison mechanism is integrated to iteratively eliminate inferior designs. The method achieves high reliability while drastically reducing the number of model evaluations, making it well-suited for large-scale engineering applications where computational efficiency and decision accuracy must be carefully balanced.

Bayesian calibrationBayesian optimal experimental designexpensive computational models

Hot Scholars

AM

Alexander Munteanu

TU Dortmund University
algorithmsmachine learningoptimizationtheoretical computer science
YY

Yuhao Yi

Sichuan University
Optimization and ControlNetworksMachine LearningBioinformatics
JF

Jing Fu

Professor of Ophthalmology, Beijing Tongren Hospital, Capital Medical University, Beijing, China
Clinician ScientistOphthalmology
EH

Eyke Hüllermeier

Professor of Computer Science, Paderborn University
Artificial IntelligenceMachine LearningFuzzy LogicBioinformatics