Solomonoff induction

2026-03-17

AI Summary

This study investigates whether an idealized inductive method for universal prediction can simultaneously satisfy two natural computability requirements, thereby providing a theoretical foundation for Occam’s razor. Formalizing this framework within computability theory, the authors employ a diagonalization argument—extending Putnam’s approach—to rigorously analyze the limitations of Solomonoff induction. The analysis demonstrates that Solomonoff induction cannot fulfill both computability conditions concurrently, challenging its prevailing status as the canonical ideal model for universal prediction and as a formal justification for Occam’s razor. This result exposes fundamental computability-theoretic constraints inherent in current idealized models of learning, suggesting that any such model aspiring to universality must reconcile these intrinsic limitations.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Other Foundations of Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Models for Web evolution

Abstract

This chapter discusses the Solomonoff approach to universal prediction. The crucial ingredient in the approach is the notion of computability, and I present the main idea as an attempt to meet two plausible computability desiderata for a universal predictor. This attempt is unsuccessful, which is shown by a generalization of a diagonalization argument due to Putnam. I then critically discuss purported gains of the approach, in particular it providing a foundation for the methodological principle of Occam's razor, and it serving as a theoretical ideal for the development of machine learning methods.

Problem

• Solomonoff induction
• universal prediction
• computability
• diagonalization
• Occam's razor

Innovation

• Solomonoff induction
• computability
• diagonalization argument
• Occam's razor
• universal prediction