Fast Semiparametric Density Regression with Weight-localized Predictive Recursion

2026-05-03

AI Summary

This work addresses the computational challenges of traditional Bayesian methods for mixture density estimation in regression settings with covariate-dependent distributions, which suffer from high computational costs due to the absence of recursive updating mechanisms. The authors propose the PRx algorithm, which extends Predictive Recursion (PR) to regression for the first time. By incorporating kernel-weighted localization, PRx overcomes the non-recursive nature of dependent Dirichlet processes, yielding a nonparametric, consistent estimator with linear computational complexity. Integrating kernel smoothing, the PR recursive framework, and a PRMLx likelihood scoring criterion, the method achieves accuracy comparable to state-of-the-art Bayesian approaches in simulations and real-data analyses—within seconds to minutes—dramatically outpacing Markov chain Monte Carlo methods that require hours. PRx further supports extensions such as model comparison and multiple testing for covariate dependence.

Abstract

Predictive recursion (PR) is a fast algorithm for nonparametric estimation of a mixing density, with connections to sequential Bayesian updating under a Dirichlet process prior and rigorous frequentist consistency guarantees. Extending PR to the regression setting, where one seeks to estimate how a mixing density varies with covariate, is nontrivial: dependent Dirichlet process priors, the natural Bayesian generalization, gives no simple recursive updating formula. We introduce PRx, which overcomes this challenge through combining kernel-based weight localization with the recursive scheme of the original PR algorithm. The algorithm scales linearly in sample size and covariate dimension, completing in seconds to minutes where MCMC-based competitors require hours. Exactly as with ordinary PR, the algorithm produces as a byproduct a likelihood score, the PRMLx, whose maximizer is shown to be a consistent estimator for unmixed parameters. In simulations and case studies PRx produces conditional density estimates competitive with established Bayesian procedures at a fraction of the computational cost, and can also be adapted for a wide range of statistical applications including Bayesian model comparison and covariate-dependent multiple testing.

Problem

• density regression
• mixing density
• covariate-dependent
• semiparametric estimation
• nonparametric estimation

Innovation

• Predictive Recursion
• Semiparametric Density Regression
• Weight Localization
• Dirichlet Process
• Scalable Inference