analytical marginalization

This work addresses the challenge of efficiently estimating and optimizing the marginal likelihood and its functionals in models with high-dimensional parameters and low-dimensional continuous hyperparameters. The authors propose a smooth functional estimation framework based on posterior samples, which integrates Bayesian posterior sampling, function interpolation, and functional estimation techniques. By leveraging posterior samples evaluated at a finite set of grid points, the method constructs a globally smooth estimator of the marginal likelihood. Notably, it unifies sequential Monte Carlo, Gibbs sampling, and Monte Carlo maximum likelihood approaches within a single theoretical framework and establishes estimation consistency under both fixed and dense grid designs. Empirical evaluations on Gaussian process regression and classification, as well as cross-effects models, demonstrate that the approach accurately estimates the marginal likelihood and its associated functionals—including derivatives and integrals.

This study investigates the frequentist validity of two-step (plug-in) approaches in semiparametric Bayesian inference, with particular emphasis on settings involving nuisance parameters. For models satisfying Neyman orthogonality conditions, the authors demonstrate that marginal posteriors for the target parameter retain desirable frequentist properties—even when uncertainty in estimating the nuisance parameters is ignored—by effectively severing feedback between the nuisance and target parameters. The analysis is further extended to non-orthogonal settings, where posterior asymptotic robustness is guaranteed under mere consistency of the nuisance parameter estimator. Methodologically, the framework combines Dirichlet processes with Bayesian bootstrap techniques for nonparametric modeling and is applied to plug-in estimation of propensity scores in causal inference, showing that the plug-in step exerts negligible influence on the resulting posterior for the target parameter.

In Bayesian sequential inference, the marginal likelihood is often treated as a static constant, overlooking its role in modulating the pace of belief updates. This work reveals that the marginal likelihood not only governs the magnitude of individual updates but also encodes frequency patterns embedded in historical data, which the authors reformulate as a dynamic regularizer. By introducing three diagnostic metrics to control online estimation gain and integrating prior and posterior distributions into a hybrid probabilistic mechanism, the proposed approach adaptively adjusts to distributional drift. The resulting framework unifies Bayesian updating with frequentist characteristics within a two-layer probabilistic architecture, substantially enhancing the robustness of sequential estimation and offering a novel paradigm for online risk quantification.

This work addresses the challenge of marginal likelihood (evidence) estimation in likelihood-free inference, where the likelihood function is intractable. We propose the first method to directly approximate the marginal likelihood from the output of sequential neural likelihood estimation (SNLE), without requiring additional simulations or posterior sampling. Leveraging only the density-ratio estimates obtained during SNLE training, our approach constructs an efficient and broadly applicable evidence estimator. Crucially, this is the first work to systematically integrate SNLE with Bayesian model comparison—bridging a key gap in simulation-based inference (SBI). It achieves improved estimation reliability over existing SBI model selection methods while preserving computational efficiency. Extensive experiments on multiple benchmark tasks demonstrate that our estimator closely approximates the true marginal likelihood. This advances both the theoretical foundations and practical applicability of neural density estimation–based inference for model selection.

Constructing efficient debiased estimators traditionally requires manual derivation of the efficient influence function (EIF), a labor-intensive process with high technical barriers and poor scalability. Method: This paper introduces Dimple, the first framework that models statistical functionals as compositions of differentiable primitives satisfying a novel differentiability condition; it leverages automatic differentiation to directly generate unbiased, efficient estimators while simultaneously identifying nuisance parameters. Dimple integrates probabilistic programming with functional decomposition, eliminating the need for explicit EIF derivation. Contribution/Results: We provide an open-source Python library enabling users to define parameters, generate estimators, and perform inference in just a few lines of code. Extensive experiments demonstrate Dimple’s effectiveness across diverse causal and semiparametric models—including AIPW, DR-Learner, and doubly robust IV—significantly lowering the barrier to constructing efficient estimators without sacrificing statistical efficiency.

This work addresses the lack of explicit confidence modeling for various sources of uncertainty in Bayesian inference by proposing a general extension framework that, for the first time, explicitly incorporates confidence in key uncertainty components—such as the prior and likelihood—into Bayesian modeling. The framework not only introduces a novel regularization mechanism but also provides a unified approach to inducing model sparsity. Without compromising theoretical rigor, the method achieves controllable sparsity across diverse models, including linear regression, logistic regression, and Bayesian neural networks, thereby significantly enhancing both interpretability and generalization performance.

This work addresses the significant bias introduced by Integrated Laplace Approximation (ILA) in latent-variable Gaussian models, which often leads to posterior distributions that deviate substantially from the true posterior and degrade downstream task performance. To mitigate this issue, the authors propose a correction framework based on importance sampling that integrates pseudo-marginalization with quasi-Monte Carlo methods within an automatic differentiation environment, enabling gradient-based hyperparameter inference. The approach innovatively combines randomized quasi-Monte Carlo with Hamiltonian Monte Carlo to construct an error-correction mechanism provably convergent to the true posterior. Experimental results demonstrate that the proposed method substantially reduces approximation error across a range of practical models and significantly enhances the accuracy of Bayesian inference.

This study addresses Bayesian inference for low-dimensional target parameters in semiparametric models, particularly under the presence of complex nuisance components that may compromise frequentist properties. To this end, we construct posterior distributions by integrating estimating function methods with nonparametric Bayesian techniques—such as Dirichlet processes and Bayesian bootstrap—under conditions weaker than the classical stochastic equicontinuity assumption. We establish asymptotic normality and consistency of the resulting posterior, rigorously identifying the key assumptions required to guarantee desirable frequentist behavior. The theoretical analysis systematically elucidates how relaxing these assumptions affects inferential performance. Extensive simulations corroborate the effectiveness of the proposed methodology, demonstrating its robustness and accuracy in practical settings.