Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling

2026-10-07

AI Summary

This study addresses the efficiency-accuracy imbalance in density estimation under large-scale non-stationary kernels and high-dimensional covariates by proposing a scalable Bayesian estimator based on logistic Gaussian processes. Methodologically, it employs a separable covariance structure to model response-covariate dependencies and directly samples from the latent field, circumventing Laplace or variational approximations. The approach integrates symmetric mini-batch splitting with Kronecker-whitened coordinates to optimize Hamiltonian dynamics, while combining kinetic Langevin sampling with Fisher identities for efficient gradient computation. Experimental results demonstrate that, on a benchmark comprising 3.9 million samples, the proposed method requires only single-GPU training yet achieves density estimation and calibration metrics comparable to those of state-of-the-art tabular foundation models.

Abstract

Conditional density estimation targets the full distribution of a response given covariates, as required, for example, for per-galaxy photometric redshifts. We develop a scalable Bayesian estimator based on the logistic Gaussian process. The log conditional density has a separable covariance: a Matérn kernel along the response, represented in a truncated Fourier basis on a circle, and a covariate kernel represented by Nyström features, which accommodate non-stationary kernels with input-dependent amplitudes and length scales. Instead of a Laplace or variational approximation, we sample the latent field of this finite-feature model. Given the hyperparameters, its posterior is strongly log-concave with a uniformly bounded Hessian, and we draw from it by simulating kinetic Langevin dynamics with symmetric minibatch splitting in Kronecker-whitened coordinates. Marginal-likelihood gradients follow from Fisher's identity as posterior expectations. Under the conditions of our analysis their bias is controlled by the sampler's step size and run length, and the predictive averages over the non-Gaussian latent posterior instead of a Gaussian around its mode. On photometric-redshift benchmarks with up to 3.9 million training observations, trained on a single GPU, the estimator is competitive with state-of-the-art tabular foundation models on density and calibration metrics.

Problem

• conditional density estimation
• logistic Gaussian process
• scalability
• Bayesian inference

Innovation

• Logistic Gaussian Process
• Kinetic Langevin Sampling
• Conditional Density Estimation
• Nyström Features
• Scalable Bayesian Inference