Outrigger local polynomial regression

2026-03-11

AI Summary

This study addresses the limitations of classical local polynomial regression under heteroscedastic or non-Gaussian errors, where performance deteriorates due to its inability to adapt to the conditional error distribution. The authors propose a novel adaptive local polynomial regression method that estimates the conditional score function of the errors and incorporates an “outrigger” mechanism to achieve stable estimation over an extended bandwidth, without requiring independence between errors and covariates or symmetry of the error distribution. The method guarantees worst-case risk no worse than the standard approach, coincides with it under Gaussian errors, and attains minimax optimality—up to a factor of at most 1.69—for Hölder smooth functions. Theoretical analysis establishes its distributional adaptivity and optimality, while simulations and real-data experiments demonstrate its empirical advantages. An open-source R implementation is provided.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Local Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Standard local polynomial estimators of a nonparametric regression function employ a weighted least squares loss function that is tailored to the setting of homoscedastic Gaussian errors. We introduce the outrigger local polynomial estimator, which is designed to achieve distributional adaptivity across different conditional error distributions. It modifies a standard local polynomial estimator by employing an estimate of the conditional score function of the errors and an 'outrigger' that draws on the data in a broader local window to stabilise the influence of the conditional score estimate. Subject to smoothness and moment conditions, and only requiring consistency of the conditional score estimate, we first establish that even under the least favourable settings for the outrigger estimator, the asymptotic ratio of the worst-case local risks of the two estimators is at most $1$, with equality if and only if the conditional error distribution is Gaussian. Moreover, we prove that the outrigger estimator is minimax optimal over Hölder classes up to a multiplicative factor $A_{β,d}$, depending only on the smoothness $β\in (0,\infty)$ of the regression function and the dimension~$d$ of the covariates. When $β\in (0,1]$, we find that $A_{β,d} \leq 1.69$, with $\lim_{β\searrow 0} A_{β,d} = 1$. A further attraction of our proposal is that we do not require structural assumptions such as independence of errors and covariates, or symmetry of the conditional error distribution. Numerical results on simulated and real data validate our theoretical findings; our methodology is implemented in R and available at https://github.com/elliot-young/outrigger.

Problem

• local polynomial regression
• conditional error distribution
• distributional adaptivity
• nonparametric regression
• heteroscedasticity

Innovation

• outrigger estimator
• distributional adaptivity
• local polynomial regression
• conditional score function
• minimax optimality