🤖 AI Summary
This paper investigates the minimax risk of score function estimation under log-concave distributions, focusing on the joint impact of tail behavior and smoothness on estimation accuracy. We introduce a novel density class that simultaneously imposes quantitative tail control and $(eta, L)$-Hölder smoothness constraints. For this class, we establish the optimal convergence rate—$n^{-eta/(2eta+1)}$, up to logarithmic factors, in the $L^2$ norm—for score function estimation—a first such result. Theoretically, this rate is strictly faster than those attainable under shape constraints alone or smoothness constraints alone when $eta in [1,2)$, revealing a synergistic gain from their combination. Methodologically, we construct a locally adaptive multiscale estimator based on uniform confidence bands, enabling automatic adaptation to heterogeneous smoothness across the domain. Our results unify and extend the theoretical frontiers of both shape-constrained estimation and nonparametric smooth function estimation.
📝 Abstract
Score estimation has recently emerged as a key modern statistical challenge, due to its pivotal role in generative modelling via diffusion models. Moreover, it is an essential ingredient in a new approach to linear regression via convex $M$-estimation, where the corresponding error densities are projected onto the log-concave class. Motivated by these applications, we study the minimax risk of score estimation with respect to squared $L^2(P_0)$-loss, where $P_0$ denotes an underlying log-concave distribution on $mathbb{R}$. Such distributions have decreasing score functions, but on its own, this shape constraint is insufficient to guarantee a finite minimax risk. We therefore define subclasses of log-concave densities that capture two fundamental aspects of the estimation problem. First, we establish the crucial impact of tail behaviour on score estimation by determining the minimax rate over a class of log-concave densities whose score function exhibits controlled growth relative to the quantile levels. Second, we explore the interplay between smoothness and log-concavity by considering the class of log-concave densities with a scale restriction and a $(β,L)$-Hölder assumption on the log-density for some $βin [1,2]$. We show that the minimax risk over this latter class is of order $L^{2/(2β+1)}n^{-β/(2β+1)}$ up to poly-logarithmic factors, where $n$ denotes the sample size. When $β< 2$, this rate is faster than could be obtained under either the shape constraint or the smoothness assumption alone. Our upper bounds are attained by a locally adaptive, multiscale estimator constructed from a uniform confidence band for the score function. This study highlights intriguing differences between the score estimation and density estimation problems over this shape-constrained class.