Empirical Measures and Strong Laws of Large Numbers in Categorical Probability

2025-03-27

AI Summary

This paper addresses the unified characterization of the limiting behavior of empirical measures in categorical probability theory. Method: It introduces two axioms—permutation invariance and empirical sufficiency—to axiomatize the “empirical sampling” operation—i.e., the partial morphism mapping infinite random sequences to empirical measures—within a categorical framework. To model ill-defined sequences, it constructs a quasi-Markov category, employing partial Markov kernels on standard Borel spaces, and integrates categorical semantics with probabilistic logic axiomatization techniques. Contribution/Results: From this single axiomatization, it uniformly derives abstract categorical versions of de Finetti’s theorem, the Glivenko–Cantelli theorem, and the strong law of large numbers. Moreover, it rigorously recovers their classical formulations, establishing representability and structural uniqueness at the categorical level—thereby achieving a first-principles unification of empirical limit theory.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The Glivenko-Cantelli theorem is a uniform version of the strong law of large numbers. It states that for every IID sequence of random variables, the empirical measure converges to the underlying distribution (in the sense of uniform convergence of the CDF). In this work, we provide tools to study such limits of empirical measures in categorical probability. We propose two axioms, permutation invariance and empirical adequacy, that a morphism of type $X^mathbb{N} o X$ should satisfy to be interpretable as taking an infinite sequence as input and producing a sample from its empirical measure as output. Since not all sequences have a well-defined empirical measure, ``such empirical sampling morphisms'' live in quasi-Markov categories, which, unlike Markov categories, allow partial morphisms. Given an empirical sampling morphism and a few other properties, we prove representability as well as abstract versions of the de Finetti theorem, the Glivenko-Cantelli theorem and the strong law of large numbers. We provide several concrete constructions of empirical sampling morphisms as partially defined Markov kernels on standard Borel spaces. Instantiating our abstract results then recovers the standard Glivenko-Cantelli theorem and the strong law of large numbers for random variables with finite first moment. Our work thus provides a joint proof of these two theorems in conjunction with the de Finetti theorem from first principles.

Problem

• Study limits of empirical measures in categorical probability
• Propose axioms for empirical sampling morphisms in quasi-Markov categories
• Prove abstract versions of de Finetti, Glivenko-Cantelli, and strong law of large numbers

Innovation

• Introduces quasi-Markov categories for partial morphisms
• Proposes axioms for empirical sampling morphisms
• Provides concrete constructions using Markov kernels