The Problem of the Priors, or Posteriors?

2025-03-14

AI Summary

This paper addresses the “posterior problem” in Bayesian inference: how to formally constrain priors so that posterior beliefs converge to the true parameter value as data accumulate. To this end, we propose **forward Bayesianism**—a framework that takes **posterior consistency as the foundational normative criterion**, and derives necessary conditions for prior admissibility *retroactively* from this requirement. This approach is the first to systematically establish asymptotic convergence as a core normative principle in epistemology. Integrating Bayesian probability theory, the principle of conditionalization, and asymptotic analysis, forward Bayesianism reconceptualizes the Bayesian foundations of Occam’s razor. It thereby provides a novel normative basis for model selection in statistical inference and machine learning, and substantially strengthens the theoretical justification for parsimony.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The problem of the priors is well known: it concerns the challenge of identifying norms that govern one's prior credences. I argue that a key to addressing this problem lies in considering what I call the problem of the posteriors -- the challenge of identifying norms that directly govern one's posterior credences, which then induce constraints on the priors via the diachronic requirement of conditionalization. This forward-looking approach can be summarized as: Think ahead, work backward. Although this idea can be traced to Freedman (1963), Carnap (1963), and Shimony (1970), it has received little attention in philosophy. In this paper, I initiate a systematic defense of forward-looking Bayesianism, addressing potential objections from more traditional views (both subjectivist and objectivist) and arguing for its advantages. In particular, I develop a specific approach to forward-looking Bayesianism -- one that treats the convergence of posterior credences to the truth as a fundamental rather than derived normative requirement. This approach, called convergentist Bayesianism, is argued to be crucial for a Bayesian foundation of Ockham's razor and related inference methods in statistics and machine learning.

Problem

• Identifying norms governing prior credences in Bayesianism.
• Exploring norms for posterior credences via conditionalization.
• Developing convergentist Bayesianism for truth convergence and Ockham's razor.

Innovation

• Forward-looking Bayesianism approach
• Convergentist Bayesianism normative requirement
• Diachronic conditionalization constraints on priors