🤖 AI Summary
This paper addresses the challenge of posterior inference in large-scale nonparametric regression. We propose a spatially adaptive distributed Gaussian process (GP) approximation method. The approach partitions the input space into disjoint subsets, fits independent GP posteriors on each subset using a Matérn kernel and an integrated Brownian motion prior, and incorporates a Bayesian prior on the length scale to enhance regularization of local smoothness. A novel weighted spatial aggregation scheme is then introduced to fuse these sub-posteriors into a global approximation of the full-data posterior. Theoretically, we establish that the resulting approximate posterior achieves a convergence rate that automatically adapts to the local smoothness of the true regression function—matching the minimax optimal rate attainable under the full-data setting. Empirically, our method significantly outperforms existing distributed GP approaches on both synthetic and real-world datasets, while effectively capturing heterogeneous local regularity and overcoming the smoothness rigidity inherent in standard GP models.
📝 Abstract
We consider the accuracy of an approximate posterior distribution in nonparametric regression problems by combining posterior distributions computed on subsets of the data defined by the locations of the independent variables. We show that this approximate posterior retains the rate of recovery of the full data posterior distribution, where the rate of recovery adapts to the smoothness of the true regression function. As particular examples we consider Gaussian process priors based on integrated Brownian motion and the Mat'ern kernel augmented with a prior on the length scale. Besides theoretical guarantees we present a numerical study of the methods both on synthetic and real world data. We also propose a new aggregation technique, which numerically outperforms previous approaches. Finally, we demonstrate empirically that spatially distributed methods can adapt to local regularities, potentially outperforming the original Gaussian process.
