LASER: A new method for locally adaptive nonparametric regression

2024-12-27

AI Summary

To address insufficient local adaptivity in nonparametric regression, this paper proposes LASER: a variable-bandwidth local polynomial regression framework. Methodologically, LASER achieves pointwise optimal local adaptation—matching the local Hölder regularity of the regression function at each domain point—under a single global tuning parameter, without requiring prespecified model structure. Theoretically, LASER attains the local minimax optimal convergence rate, providing rigorous statistical optimality guarantees. Algorithmically, it integrates local polynomial fitting, data-driven bandwidth selection, and adaptive estimation of the local Hölder exponent. Numerical experiments demonstrate that LASER significantly outperforms existing locally adaptive methods across diverse smoothness-heterogeneous settings, while maintaining computational efficiency. By unifying strong theoretical foundations with practical implementability, LASER fills a critical gap in the literature on locally adaptive nonparametric regression.

Technology Category

• Search and Optimization: Local Search
• Machine Learning: Online Learning & Bandits
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this article, we introduce extsf{LASER} (Locally Adaptive Smoothing Estimator for Regression), a computationally efficient locally adaptive nonparametric regression method that performs variable bandwidth local polynomial regression. We prove that it adapts (near-)optimally to the local H""{o}lder exponent of the underlying regression function exttt{simultaneously} at all points in its domain. Furthermore, we show that there is a single ideal choice of a global tuning parameter under which the above mentioned local adaptivity holds. Despite the vast literature on nonparametric regression, instances of practicable methods with provable guarantees of such a strong notion of local adaptivity are rare. The proposed method achieves excellent performance across a broad range of numerical experiments in comparison to popular alternative locally adaptive methods.

Problem

• Flexible Regression
• Local Data Adaptation
• Model-free Approach

Innovation

• LASER
• Flexible Local Adaptation
• Optimal Performance