Semi-parametric Bayesian inference under Neyman orthogonality

2026-02-23

AI Summary

This study investigates the frequentist validity of two-step (plug-in) approaches in semiparametric Bayesian inference, with particular emphasis on settings involving nuisance parameters. For models satisfying Neyman orthogonality conditions, the authors demonstrate that marginal posteriors for the target parameter retain desirable frequentist properties—even when uncertainty in estimating the nuisance parameters is ignored—by effectively severing feedback between the nuisance and target parameters. The analysis is further extended to non-orthogonal settings, where posterior asymptotic robustness is guaranteed under mere consistency of the nuisance parameter estimator. Methodologically, the framework combines Dirichlet processes with Bayesian bootstrap techniques for nonparametric modeling and is applied to plug-in estimation of propensity scores in causal inference, showing that the plug-in step exerts negligible influence on the resulting posterior for the target parameter.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Bayesian Learning
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

The validity of two-step or plug-in inference methods is questioned in the Bayesian framework. We study semi-parametric models where the plug-in of a non-parametrically modelled nuisance component is used. We show that when the nuisance and targeted parameters satisfy a Neyman orthogonal score property, the approach of cutting feedback through a two-step procedure is a valid way of conducting Bayesian inference. Our method relies on a non-parametric Bayesian formulation based on the Dirichlet process and the Bayesian bootstrap. We show that the marginal posterior of the targeted parameter exhibits good frequentist properties despite not accounting for the inferential uncertainty of the nuisance parameter. We adopt this approach in Bayesian causal inference problems where the nuisance propensity score model is estimated to obtain marginal inference for the treatment effect parameter, and demonstrate that a plug-in of the propensity score has a negligible effect on marginal posterior inference for the causal contrast. We investigate the absence of Neyman orthogonality and exploit our findings to show that in conventional two-step procedures, the posterior distribution converges under weaker restrictions than those needed in the frequentist sequel. For a simple family of useful scores, we demonstrate that even in the absence of Neyman orthogonality, the posterior distribution is asymptotically unchanged by the estimation of the nuisance parameter, merely provided the latter estimator is consistent.

Problem

• semi-parametric Bayesian inference
• Neyman orthogonality
• nuisance parameter
• plug-in inference
• causal inference

Innovation

• Neyman orthogonality
• semi-parametric Bayesian inference
• two-step procedure
• Dirichlet process
• Bayesian causal inference