Algorithmic randomness and the weak merging of computable probability measures

2025-04-01

AI Summary

This paper investigates equivalences between algorithmic randomness (Martin-Löf and Schnorr randomness) and weak convergence of probability measures, with emphasis on convergence characterizations for finite-time events. Methodologically, it jointly analyzes total variation distance, Hellinger distance, and Kullback–Leibler (KL) divergence, establishing—*for the first time*—an equivalence characterization of randomness via summable KL divergence. It further links the predictable increments in Doob decomposition to the dynamic evolution of KL divergence, thereby uncovering an information-theoretic mechanism underlying randomness. Finally, it provides a global extension of Vovk’s local theorem. The contributions unify the information-theoretic foundations of algorithmic randomness and learning-theoretic convergence, bridging computable probability theory and stochastic processes. The results yield a novel paradigm for analyzing randomness through measurable information dynamics, with implications for sequential prediction, Bayesian consistency, and the computability of stochastic processes.

Technology Category

Machine Learning: Information TheoryReasoning under Uncertainty: Stochastic OptimizationKnowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSemantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semanticsWeb Mining and Content Analysis: Models for Web evolution

Abstract

We characterize Martin-L""of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus on finite horizon events, that is, on weak merging in the sense of Kalai-Lehrer. In contrast to Blackwell-Dubins and Kalai-Lehrer, we consider not only the total variational distance but also the Hellinger distance and the Kullback-Leibler divergence. Our main result is a characterization of Martin-L""of randomness and Schnorr randomness in terms of weak merging and the summable Kullback-Leibler divergence. The main proof idea is that the Kullback-Leibler divergence between $mu$ and $ u$, at a given stage of the learning process, is exactly the incremental growth, at that stage, of the predictable process of the Doob decomposition of the $ u$-submartingale $L(sigma)=-ln frac{mu(sigma)}{ u(sigma)}$. These characterizations of algorithmic randomness notions in terms of the Kullback-Leibler divergence can be viewed as global analogues of Vovk's theorem on what transpires locally with individual Martin-L""of $mu$- and $ u$-random points and the Hellinger distance between $mu, u$.

Problem

• Characterize Martin-Löf and Schnorr randomness via merging opinions
• Compare merging using variational, Hellinger, and Kullback-Leibler distances
• Link algorithmic randomness to Kullback-Leibler divergence in learning processes

Innovation

• Characterizes randomness using merging opinions framework
• Uses multiple distance metrics for weak merging
• Links Kullback-Leibler divergence to algorithmic randomness