Nonsmooth Nonparametric Regression via Fractional Laplacian Eigenmaps

2024-02-22

AI Summary

Modeling real-world regression functions that are nonsmooth—specifically, those residing in the $L^2$ fractional Sobolev space of order $s in (0,1)$—poses a fundamental challenge for conventional nonparametric methods requiring higher-order differentiability. Method: This paper proposes a novel nonparametric regression framework based on the fractional Laplacian operator. It introduces, for the first time, fractional Laplacian feature mapping into nonparametric regression, circumventing restrictive smoothness assumptions and naturally accommodating canonical nonsmooth structures such as piecewise constants, sharp peaks, and fractional powers. The method integrates fractional differential operators, Sobolev space theory, and spectral regularization. Results: It achieves the minimax-optimal convergence rate $n^{-2s/(2s+d)}$, with a tight theoretical error bound. Numerical experiments demonstrate its substantial superiority over classical smooth kernel methods in fitting nonsmooth functions.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Kernel Methods
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We develop nonparametric regression methods for the case when the true regression function is not necessarily smooth. More specifically, our approach is using the fractional Laplacian and is designed to handle the case when the true regression function lies in an $L_2$-fractional Sobolev space with order $sin (0,1)$. This function class is a Hilbert space lying between the space of square-integrable functions and the first-order Sobolev space consisting of differentiable functions. It contains fractional power functions, piecewise constant or polynomial functions and bump function as canonical examples. For the proposed approach, we prove upper bounds on the in-sample mean-squared estimation error of order $n^{-frac{2s}{2s+d}}$, where $d$ is the dimension, $s$ is the aforementioned order parameter and $n$ is the number of observations. We also provide preliminary empirical results validating the practical performance of the developed estimators.

Problem

• Estimating nonsmooth regression functions via fractional Laplacian
• Handling functions in L2-fractional Sobolev spaces (0<s<1)
• Proving error bounds for in-sample mean-squared estimation

Innovation

• Fractional Laplacian for nonsmooth regression
• L2-fractional Sobolev space function class
• In-sample mean-squared error bounds analysis