A domain-theoretic framework for conditional probability and Bayesian updating in programming

2025-02-02

AI Summary

Probabilistic programming lacks a computable definition of conditional probability and a rigorous semantic foundation for Bayesian updating. Method: This paper establishes the first domain-theoretic framework for probabilistic programming, introducing a constructive, observation-based definition of conditional probability that is computably realizable. It unifies and proves the semantic equivalence of density-based methods and rejection sampling over trajectories within this framework. Furthermore, it designs a functional probabilistic language equipped with sampling and evaluation primitives, and develops both operational and denotational semantics, verifying their consistency. Contribution/Results: The work provides the first computable and verifiable theoretical foundation for conditional inference in probabilistic programming, enabling rigorous modeling and analysis of Bayesian updates. It bridges foundational probability theory with practical probabilistic computation, supporting formal verification and semantic reasoning about probabilistic programs.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Machine Learning: Probabilistic Circuits and Graphical Models
• Knowledge Representation and Reasoning: Logic Programming

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a domain-theoretic framework for probabilistic programming that provides a constructive definition of conditional probability and addresses computability challenges previously identified in the literature. We introduce a novel approach based on an observable notion of events that enables computability. We examine two methods for computing conditional probabilities -- one using conditional density functions and another using trace sampling with rejection -- and prove they yield consistent results within our framework. We implement these ideas in a simple probabilistic functional language with primitives for sampling and evaluation, providing both operational and denotational semantics and proving their consistency. Our work provides a rigorous foundation for implementing conditional probability in probabilistic programming languages.

Problem

• Probabilistic Programming
• Bayesian Inference
• Complex Mathematical Problems

Innovation

• Probabilistic Programming
• Conditional Probability Computation
• Theoretical Consistency