cross-fitting procedures

This work addresses the challenges of achieving efficient semiparametric estimation of low-dimensional target parameters in the presence of high-dimensional nuisance functions, where out-of-sample prediction of nuisances and statistical efficiency are critical concerns. The authors propose a general, estimator-agnostic cross-fitting engine that automatically executes reproducible folding schedules based on user-specified target functionals and a directed acyclic graph (DAG) encoding the nuisance model structure. Innovatively leveraging graph-based nuisance modeling, the framework supports folding strategies such as disjointness and independence enhancement, while reducing branching dependencies through node replication. It further incorporates explicit scheduling, dependency validation, intelligent caching, and fault isolation mechanisms. Implemented as a lightweight R package publicly available on CRAN, this approach substantially improves the controllability, reproducibility, and computational efficiency of cross-fitting estimators, facilitating large-scale simulations and rapid prototyping.

This paper addresses robust inference for a low-dimensional target parameter β in the presence of infinite-dimensional nuisance parameters. Conventional double machine learning (DML) requires nuisance estimators to converge at a rate faster than n⁻¹/⁴; otherwise, Wald-type confidence intervals become invalid. To overcome this limitation, we propose Perturbed Double Machine Learning (P-DML): it injects random perturbations into nuisance estimation to generate multiple β estimates, then applies bias screening and thresholding to retain only valid ones—thereby relaxing the n⁻¹/⁴ rate requirement. P-DML accommodates both Lasso and general machine learning algorithms without imposing strong convergence conditions on nuisance estimators. We establish that the resulting estimator possesses oracle properties and yields asymptotically valid confidence intervals. Simulation studies demonstrate that P-DML maintains nominal coverage even when nuisance parameters converge slowly, substantially enhancing the robustness of statistical inference.

This study addresses the severe undercoverage of confidence intervals when estimating population average treatment effects using flexible machine learning models under complex survey designs, such as stratified multistage sampling. The authors propose a survey-weighted targeted maximum likelihood estimator (TMLE) with cross-fitting implemented at the primary sampling unit (PSU) level, combined with Taylor linearization of the influence function to obtain design-consistent variance estimates. Theory and simulations demonstrate for the first time that valid inference requires cross-fitting specifically at the PSU level—internal cross-validation is insufficient. In NHANES-like simulations, the proposed method achieves stable coverage of 93%–95%, substantially outperforming single-fit TMLE (as low as 22%) and internal cross-validation (85%–88%). Empirical analyses of four NHANES datasets confirm its practical utility, and accompanying open-source software has been released.

This work addresses the suboptimal inference in semiparametric estimation caused by estimation errors in nuisance functions when using black-box machine learning models. The authors propose a novel estimator that, without imposing additional assumptions, eliminates first-order stochastic errors from nuisance estimation and achieves optimal convergence rates even when auxiliary functions cannot be consistently estimated. Built upon the framework of orthogonal scores and semiparametric linear functionals, the proposed estimator attains the sharp rate \(n^{-1/2} + \delta^a_\mu + (\delta^s_\mu)^2\) and is shown to be asymptotically normal with minimal asymptotic variance. Its tuning strategy favors undersmoothing and substantially outperforms classical double machine learning methods, making it well-suited for widespread applications such as average treatment effect estimation.

This paper addresses robust estimation of average partial effects (APEs) in nonlinear models under moderate-dimensional settings. We propose a novel double machine learning framework that dispenses with linearity assumptions and differentiability requirements on the regression model, permitting arbitrary black-box machine learning algorithms as first-stage estimators. Our method innovatively introduces re-smoothing to confer differentiability upon otherwise non-differentiable estimators; integrates a location-scale model to flexibly characterize the conditional distribution of covariates; and constructs a doubly robust semiparametric inference procedure. We establish theoretical guarantees: the estimator achieves the semiparametric efficiency bound and remains robust under model misspecification and other nonstandard conditions. Numerical experiments demonstrate substantial improvements over existing APE estimators in both estimation accuracy and confidence interval coverage.

In causal machine learning, it remains unclear whether standard cross-fitting can effectively eliminate bias introduced by black-box algorithms when observational units exhibit spatial, clustered, or time-series dependencies. This study systematically evaluates the performance of conventional cross-fitting that ignores such dependence structures through theoretical analysis and simulation experiments. The findings reveal that, even without explicitly modeling inter-unit dependencies, standard cross-fitting successfully removes the dominant bias term and yields estimation bias and precision comparable to—or sometimes better than—specialized decorrelated folding strategies across a range of correlated data-generating mechanisms. These results challenge the prevailing assumption in the literature that customized cross-fitting procedures are necessary for dependent data, offering theoretical justification for simplifying causal inference pipelines.

This paper addresses the pervasive coverage miscalibration problem in quantile regression—i.e., the discrepancy between nominal and actual coverage probabilities of prediction intervals. We propose a model-agnostic, computationally efficient calibration framework. Its core innovation lies in establishing a theoretical connection between the leave-one-out coverage probability and the fitted values of dual variables in the quantile regression dual optimization problem, yielding an analytically tractable and numerically simple correction formula. Integrating dual optimization, leave-one-out analysis, and cross-validation, our method achieves consistent estimation and exact coverage calibration under proportional asymptotics, requiring only minimal regularity assumptions. We rigorously prove the asymptotic unbiasedness and consistency of the calibrated quantile intervals. Extensive experiments—including high-dimensional simulations and diverse real-world datasets—demonstrate the method’s robustness and substantially improved calibration accuracy over state-of-the-art alternatives.

This study addresses the instability of machine learning predictions caused by variations in random seeds, which undermines the reproducibility of debiased estimators. The authors formally introduce, for the first time, a stability condition with respect to random seeds and propose an adaptive cross-bagging method that integrates subsample bagging with cross-fitting. This approach ensures stability for any regression algorithm with bounded outputs while substantially reducing computational overhead. By simultaneously eliminating seed dependence in both perturbation-based estimation and sample splitting, the method achieves the desired level of stability with markedly higher computational efficiency than existing alternatives, as demonstrated in empirical evaluations.

Existing heterogeneous treatment effect (HTE) estimators lack robust, comparable evaluation criteria. Method: We propose the first relative-error-based robust evaluation framework for HTE estimation, overcoming the limitations of conventional absolute-error metrics. Theoretically, we establish the identifiability conditions for nuisance parameters and prove asymptotic unbiasedness of relative-error estimation. Methodologically, we design a dedicated loss function and neural architecture that jointly enable performance quantification and end-to-end learning—supporting both comparative assessment of existing estimators and optimization of novel algorithms. Results: Extensive experiments across multiple benchmark datasets demonstrate that our framework reliably discriminates estimator quality. The proposed learning algorithm consistently outperforms state-of-the-art methods—including R-Learner and DR-Learner—achieving statistically significant improvements. This work provides a principled, reproducible foundation for HTE model selection, validation, and development.

This work addresses the challenge of constructing valid prediction intervals for counterfactual outcomes under runtime confounding, where only a subset of confounding variables is observed in the target population. Existing conformal prediction methods often fail to achieve nominal coverage in such settings. To overcome this limitation, the paper introduces semi-parametric efficiency theory into the conformal prediction framework, integrating debiased machine learning with counterfactual modeling to produce prediction intervals that maintain valid coverage despite missing confounders. The proposed method not only resolves the coverage failure induced by unobserved confounding but also attains faster convergence rates. Empirical evaluations on multiple synthetic and semi-synthetic datasets demonstrate that the approach consistently achieves the desired coverage levels and significantly outperforms standard conformal prediction methods.