multiply robust semiparametric estimation

This work addresses the suboptimal inference in semiparametric estimation caused by estimation errors in nuisance functions when using black-box machine learning models. The authors propose a novel estimator that, without imposing additional assumptions, eliminates first-order stochastic errors from nuisance estimation and achieves optimal convergence rates even when auxiliary functions cannot be consistently estimated. Built upon the framework of orthogonal scores and semiparametric linear functionals, the proposed estimator attains the sharp rate \(n^{-1/2} + \delta^a_\mu + (\delta^s_\mu)^2\) and is shown to be asymptotically normal with minimal asymptotic variance. Its tuning strategy favors undersmoothing and substantially outperforms classical double machine learning methods, making it well-suited for widespread applications such as average treatment effect estimation.

This paper addresses robust estimation of average partial effects (APEs) in nonlinear models under moderate-dimensional settings. We propose a novel double machine learning framework that dispenses with linearity assumptions and differentiability requirements on the regression model, permitting arbitrary black-box machine learning algorithms as first-stage estimators. Our method innovatively introduces re-smoothing to confer differentiability upon otherwise non-differentiable estimators; integrates a location-scale model to flexibly characterize the conditional distribution of covariates; and constructs a doubly robust semiparametric inference procedure. We establish theoretical guarantees: the estimator achieves the semiparametric efficiency bound and remains robust under model misspecification and other nonstandard conditions. Numerical experiments demonstrate substantial improvements over existing APE estimators in both estimation accuracy and confidence interval coverage.

Estimating causal effects via semiparametric models in multi-source heterogeneous data (e.g., cross-hospital EHRs or clinical trials) poses challenges in privacy preservation and statistical efficiency due to data silos and structural heterogeneity. Method: We propose a privacy-preserving late-fusion multi-task learning framework that integrates double machine learning with an adaptive task aggregation mechanism. It enables two-stage estimation without sharing individual-level data and employs joint estimation of nuisance parameters across tasks, leveraging their similarity. Contribution/Results: We establish theoretical guarantees showing that when nuisance parameters are sufficiently similar across tasks, the proposed estimator achieves a faster convergence rate than single-task baselines. Empirical evaluation demonstrates substantial improvements in accuracy and robustness for heterogeneous treatment effect estimation—particularly in moderate-sample regimes—while maintaining computational efficiency and strict privacy compliance.

In causal mediation analysis, conventional estimators of the mediated effect functional suffer from low accuracy and high sensitivity to misspecification of nuisance functions. To address this, we propose a bias-structure-guided two-stage framework that decouples nuisance function estimation. In Stage I, we estimate only the bias-relevant component of the mediation mechanism—rather than the full mechanism—thereby reducing model dependence. In Stage II, we introduce a nonparametric weighted balancing estimator, where weights are constructed by directly optimizing the asymptotic bias of the mediated effect estimator. We establish theoretical guarantees: the resulting estimator is consistent and asymptotically normal, and remains robust under partial misspecification of nuisance functions. Compared with standard approaches, our method substantially improves estimation accuracy and reliability. It provides a principled tool for mediation inference in high-dimensional settings or under model uncertainty.

This paper addresses the problem of estimating functionals of an unknown target function under a structure-agnostic setting—where no specific structural assumptions (e.g., Hölder smoothness) are imposed on the nuisance function, and only a generic convergence rate for nuisance estimation is assumed. Methodologically, it introduces the first formal framework for structure-agnostic estimation, operating under three simultaneous constraints: weak regularity conditions, compatibility with general-purpose nuisance estimators, and sample splitting. Theoretically, it establishes, for the first time, the essential optimality of first-order debiased estimators in this setting. Through minimax lower bound analysis, higher-order perturbation theory, and a unified debiasing framework, the paper precisely characterizes the optimal convergence rate and quantifies the fundamental trade-off between incorporating structural priors and improving estimation efficiency. These results provide foundational theoretical support for nonparametric and semiparametric inference.

This study addresses the challenge of constructing Neyman-orthogonal scores for robust causal inference in semiparametric models with infinite-dimensional nuisance parameters. The authors propose a general framework that, for the first time, explicitly constructs orthogonal scores for a broad class of such models, yielding estimators of the target parameter that are asymptotically normal and require only a convergence rate of $o_p(n^{-1/4})$ or better for the nuisance parameter estimates. The approach seamlessly integrates with machine learning algorithms and is applied to estimate causal effects under binary instrumental variables. Numerical experiments demonstrate substantial finite-sample improvements over naive estimators, and an empirical analysis of the Oregon Health Insurance Experiment confirms the method’s robustness and practical utility in real-world settings.

This work addresses the lack of intuitive geometric interpretation in classical semiparametric efficiency theory, which has hindered the derivation and understanding of influence functions. The paper reformulates the theory within a differential geometric framework on the space of probability distributions, drawing an analogy to multivariate calculus: statistical paths, scores, and influence functions correspond respectively to curves, velocity vectors, and gradients. It demonstrates that the efficient influence function arises naturally as an orthogonal projection. By integrating functional analysis, differential geometry, and statistical inference, the study establishes a unified geometric interpretation of scores, tangent spaces, nuisance tangent spaces, and efficient influence functions. This synthesis not only clarifies several foundational theoretical issues but also substantially enhances the interpretability of methods in causal inference and missing data analysis.

This study investigates the frequentist validity of two-step (plug-in) approaches in semiparametric Bayesian inference, with particular emphasis on settings involving nuisance parameters. For models satisfying Neyman orthogonality conditions, the authors demonstrate that marginal posteriors for the target parameter retain desirable frequentist properties—even when uncertainty in estimating the nuisance parameters is ignored—by effectively severing feedback between the nuisance and target parameters. The analysis is further extended to non-orthogonal settings, where posterior asymptotic robustness is guaranteed under mere consistency of the nuisance parameter estimator. Methodologically, the framework combines Dirichlet processes with Bayesian bootstrap techniques for nonparametric modeling and is applied to plug-in estimation of propensity scores in causal inference, showing that the plug-in step exerts negligible influence on the resulting posterior for the target parameter.

This study addresses the challenge of specification testing in conditional moment models under high-dimensional nuisance parameters, where conventional approaches relying on asymptotically linear estimators struggle to accommodate modern machine learning methods. The authors propose a kernel-based locally robust testing framework that uniquely integrates Neyman-orthogonal moments, cross-fitting, and reproducing kernel Hilbert space techniques to achieve first-order insensitivity to estimation errors in nuisance parameters. Under mild convergence rates for nuisance estimators, the method ensures oracle equivalence between feasible and infeasible tests and precisely characterizes local power. Employing a fast multiplier bootstrap, the framework demonstrates excellent finite-sample performance across diverse applications—including specification tests for high-dimensional linear and logistic regression, significance testing in machine learning regression, and tests for constant conditional treatment effects—as validated by Monte Carlo simulations and empirical analysis.

In additive noise models, regression functions estimated by machine learning often induce spurious dependence between residuals and covariates, compromising the validity of downstream inference. This work proposes the first semiparametrically efficient inference method tailored to kernel-based heteroskedasticity, constructing a Hilbert space–valued one-step estimator for the kernel covariance operator between covariates and residuals. Coupled with a bootstrap calibration procedure, the approach enables valid tests for residual independence and model goodness-of-fit. The method accommodates settings with additional covariates, supports efficient inference on heterogeneity in residual noise distributions across treatment groups, and yields asymptotically valid confidence intervals. Simulations demonstrate that, compared to naive plug-in residual methods, the proposed approach achieves substantially improved calibration and statistical power.