Locally parametric nonparametric density estimation

2026-04-20

AI Summary

This study addresses the challenge of enhancing density estimation performance by integrating the structural advantages of parametric models while preserving the flexibility of nonparametric methods. The authors propose a locally parametrized nonparametric density estimator that, for each point \(x\), estimates an optimal local parameter \(\hat{\theta}(x)\) via kernel-smoothed likelihood, yielding an estimator of the form \(f(x, \hat{\theta}(x))\). This approach achieves near-full-likelihood efficiency under correct model specification and retains nonparametric robustness under misspecification, effectively serving as a semiparametric realization of higher-order kernel methods. Theoretical analysis and empirical experiments demonstrate that the proposed estimator exhibits variance comparable to classical kernel density estimation but substantially reduced bias, leading to significantly improved accuracy in neighborhoods of well-specified parametric models.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Intelligent Robots: State Estimation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

This paper develops a nonparametric density estimator with parametric overtones. Suppose $f(x,θ)$ is some family of densities, indexed by a vector of parameters $θ$. We define a local kernel smoothed likelihood function which for each $x$ can be used to estimate the best local parametric approximant to the true density. This leads to a new density estimator of the form $f(x,\hatθ(x))$, thus inserting the best local parameter estimate for each new value of $x$. When the bandwidth used is large this amounts to ordinary full likelihood parametric density estimation, while for moderate and small bandwidths the method is essentially nonparametric, using only local properties of data and the model. Alternative ways more general than via the local likelihood are also described. The methods can be seen as ways of nonparametrically smoothing the parameter within a parametric class. Properties of this new semiparametric estimator are investigated. Our preferred version has approximately the same variance as the ordinary kernel method but potentially a smaller bias. The new method is seen to perform better than the traditional kernel method in a broad nonparametric vicinity of the parametric model employed, while at the same time being capable of not losing much in precision to full likelihood methods when the model is correct. Other versions of the method are equivalent to using particular higher order kernels in a semiparametric framework. The methodology we develop can be seen as the density estimation parallel to local likelihood and local weighted least squares theory in nonparametric regression.

Problem

• density estimation
• semiparametric
• local likelihood
• nonparametric
• parametric model

Innovation

• local likelihood
• semiparametric density estimation
• kernel smoothing
• adaptive parametric approximation
• bias-variance tradeoff