Assumption-Lean Post-Integrated Inference with Negative Control Outcomes

2024-10-07

arXiv.org

AI Summary

Post-integration multiple testing is vulnerable to data-dependence bias—particularly under unmeasured confounding, latent mediation, or model misspecification. Method: We propose the first robust post-integration inference framework leveraging negative control outcomes. Our approach introduces a hypothesis-sparse, negative-control-driven paradigm and constructs a projectable direct effect estimator that is nonparametrically identifiable, robust to model misspecification, and tolerant to measurement error. It integrates causal inference, semiparametric statistics, double-robust estimation, and machine learning (e.g., random forests), supported by finite-sample linear expansion and uniform concentration bounds. Contribution/Results: The method achieves consistent and efficient estimation. Applied to single-cell CRISPR perturbation data, it successfully corrects for batch effects and unmeasured confounding. Extensive simulations and real-data analyses demonstrate its statistical superiority over existing methods.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Data integration methods aim to extract low-dimensional embeddings from high-dimensional outcomes to remove unwanted variations, such as batch effects and unmeasured covariates, across heterogeneous datasets. However, multiple hypothesis testing after integration can be biased due to data-dependent processes. We introduce a robust post-integrated inference (PII) method that adjusts for latent heterogeneity using negative control outcomes. Leveraging causal interpretations, we derive nonparametric identifiability of the direct effects, which motivates our semiparametric inference method. Our method extends to projected direct effect estimands, accounting for hidden mediators, confounders, and moderators. These estimands remain statistically meaningful under model misspecifications and with error-prone embeddings. We provide bias quantifications and finite-sample linear expansions with uniform concentration bounds. The proposed doubly robust estimators are consistent and efficient under minimal assumptions and potential misspecification, facilitating data-adaptive estimation with machine learning algorithms. Our proposal is evaluated with random forests through simulations and analysis of single-cell CRISPR perturbed datasets with potential unmeasured confounders.

Problem

• Addresses biased hypothesis testing after data integration
• Adjusts for latent heterogeneity using control outcomes
• Provides robust inference under model misspecification and errors

Innovation

• Robust post-integrated inference using control outcomes
• Nonparametric identifiability via negative control outcomes
• Semiparametric inference with surrogate control outcomes