marginal maximum likelihood

This work addresses the challenge of efficiently estimating and optimizing the marginal likelihood and its functionals in models with high-dimensional parameters and low-dimensional continuous hyperparameters. The authors propose a smooth functional estimation framework based on posterior samples, which integrates Bayesian posterior sampling, function interpolation, and functional estimation techniques. By leveraging posterior samples evaluated at a finite set of grid points, the method constructs a globally smooth estimator of the marginal likelihood. Notably, it unifies sequential Monte Carlo, Gibbs sampling, and Monte Carlo maximum likelihood approaches within a single theoretical framework and establishes estimation consistency under both fixed and dense grid designs. Empirical evaluations on Gaussian process regression and classification, as well as cross-effects models, demonstrate that the approach accurately estimates the marginal likelihood and its associated functionals—including derivatives and integrals.

This study addresses the challenge of evaluating intractable integrals or expectations by providing a systematic review of Bayesian quadrature methods. It establishes, for the first time, a unified taxonomic framework encompassing three core dimensions: modeling, inference, and sampling. Building upon Gaussian process–based probabilistic modeling and integrating Bayesian inference with numerical integration techniques, the paper elucidates the underlying mathematical foundations, synthesizes interdisciplinary literature, and assesses—through controlled numerical experiments—the impact of various design choices on empirical performance. Beyond offering a comprehensive theoretical overview and an extensive bibliography, this work critically examines practical challenges and limitations inherent in current approaches, thereby laying a cohesive foundation for future research in the field.

This study addresses the computational challenges in evaluating ground-truth causal effects—such as the average treatment effect—in simulation-based causal inference, where marginalization over confounders typically requires numerical integration. Conventional Monte Carlo methods often suffer from limited efficiency and accuracy. To overcome this, the paper systematically introduces Gaussian quadrature, particularly Gauss-Hermite quadrature, into this domain for the first time, enabling highly accurate and efficient computation of true causal effects under common confounder distributions including normal, uniform, exponential, and gamma. Across four representative simulation scenarios, the proposed approach substantially outperforms Monte Carlo integration in both precision and speed, achieving superior results with negligible computational overhead. These findings highlight the long-overlooked yet considerable utility of Gaussian quadrature in causal simulation studies.

Bayesian inference for doubly-intractable models (e.g., Ising, Kent distributions) is hindered by intractable normalizing constants in both the likelihood and posterior, rendering exact posterior computation impossible; conventional pseudo-marginal MCMC further suffers from the restrictive non-negativity requirement on the likelihood estimator. To address this, we propose a signed pseudo-marginal Metropolis–Hastings algorithm. Its core innovation is the first unbiased block-Poisson estimator that admits negative values, integrated with importance sampling correction and correlated random number techniques to circumvent the non-negativity constraint. We also derive analytical heuristic guidelines for tuning its hyperparameters. Experiments on the Ising model and spherical Kent distribution demonstrate substantial gains in effective sample size per second (ESS/sec), while preserving posterior mean consistency. This work establishes a new paradigm for efficient Bayesian inference in doubly-intractable settings.

This work addresses the fragmentation in existing Gaussian process–based approaches for approximating solutions to differential equations by proposing a unified Bayesian probabilistic framework. The framework embeds differential equation constraints into the likelihood function through derivative matching, thereby enabling simultaneous estimation of unknown parameters and quantification of uncertainty in the solution. It systematically integrates several established Gaussian process methods for the first time, elucidating their underlying connections. The generality and efficacy of the proposed approach are demonstrated across multiple benchmark problems, establishing a coherent foundation for future theoretical advancements and practical applications in this domain.

This work addresses the computational inefficiency in Bayesian semiparametric regression arising from complex design matrix structures. To mitigate this challenge, the authors propose an orthogonalization preprocessing step applied to the design日晚间 matrix prior to iterative inference, combined with a hybrid algorithm integrating Gibbs sampling and coordinate ascent variational inference. This approach reduces computational complexity to quadratic in the number of covariates, substantially accelerating both model fitting and posterior inference. Empirical evaluations across diverse experimental settings demonstrate speedups ranging from 5× to 60× compared to conventional methods, effectively alleviating the computational bottleneck induced by high-dimensional covariates.

This study investigates the statistical properties of Lagrange multipliers in constrained maximum likelihood estimation and least squares problems, along with their implications for numerical optimization. Leveraging large-sample theory, it establishes that under correctly specified models, Lagrange multipliers converge in probability to zero as the sample size grows, a result extended to high-dimensional settings such as deep learning. Building on this asymptotic behavior, the work provides the first statistical justification for initializing Lagrange multipliers at zero and integrates this insight into constrained optimization algorithms, including augmented Lagrangian methods and sequential quadratic programming. Numerical experiments demonstrate that this initialization strategy substantially enhances algorithmic stability and convergence efficiency in applications such as constrained regression and dynamic discrete choice models.

This work addresses the significant bias introduced by Integrated Laplace Approximation (ILA) in latent-variable Gaussian models, which often leads to posterior distributions that deviate substantially from the true posterior and degrade downstream task performance. To mitigate this issue, the authors propose a correction framework based on importance sampling that integrates pseudo-marginalization with quasi-Monte Carlo methods within an automatic differentiation environment, enabling gradient-based hyperparameter inference. The approach innovatively combines randomized quasi-Monte Carlo with Hamiltonian Monte Carlo to construct an error-correction mechanism provably convergent to the true posterior. Experimental results demonstrate that the proposed method substantially reduces approximation error across a range of practical models and significantly enhances the accuracy of Bayesian inference.

This study addresses the problem of efficient estimation and inference for average marginal effects in partially linear instrumental variable regression. To this end, the authors propose a one-step estimation procedure based on a reproducing kernel Hilbert space (RKHS) framework that requires only a single regularization parameter. Coupled with a Bayesian bootstrap to account for complex asymptotic variance structures, the method enables concise yet robust statistical inference. The estimator is theoretically shown to be consistent and asymptotically normal. Extensive simulations demonstrate its superior finite-sample performance, and three empirical applications yield economically meaningful results, substantially enhancing the practical utility and interpretability of semiparametric instrumental variable models.