A Theoretical Framework for Self-Play Theorem Proving Algorithms

2026-06-01

AI Summary

This work addresses the lack of theoretical characterization of self-improvement capabilities in existing self-play theorem-proving systems and their tendency to generate conjectures biased toward non-fundamental, overly complex theorems. The paper introduces the first formal theoretical framework integrating theorem semantic graphs with a self-play mechanism, enabling exponential theorem expansion through reversible random walks. It further proposes a diversity-maximizing conjecture generation algorithm based on diffusion similarity. By combining contrastive learning embeddings with inner-product similarity evaluation, the approach theoretically guarantees exponential growth of the set of proven theorems within well-connected theorem graphs, substantially enhancing both the diversity and foundational relevance of generated theorems.

Abstract

Self-play, a type of training algorithm that enables a model to self-improve, has recently shown promising empirical results in the context of formal theorem proving using Large Language Models (LLMs). (Dong & Ma, 2025) instantiate self-play with two cooperating agents: a prover, which proves theorems, and a conjecturer, which generates new theorems as a curriculum to the prover. In this paper, we provide a theoretical framework for understanding the self-improvement capabilities of self-play algorithms for theorem proving. First, we formalize the set of theorems as a graph, with nodes as theorems and edges between pairs of theorems with similar semantics. We introduce a set of primitive assumptions that characterize the guarantees of a trained prover and how a conjecturer can access the structure of the graph. Second, we show that if the underlying graph of theorems is well-connected, then a prover-conjecturer system, where the conjecturing algorithm is based on a reversible random walk, is sufficient to grow the set of proved theorems exponentially. Third, motivated by an issue encountered empirically by self-play algorithms, where the conjecturer tends to generate artificially complex and non-fundamental theorems, we propose a diversity measure for a training distribution of theorems generated by a conjecturer and an improved conjecturing algorithm that locally maximizes this diversity measure, by computing the diffusion similarity between neighboring theorems in the theorem graph. Finally, we describe a method to compute the diffusion similarity by using contrastive learning to embed nodes into Euclidean space and then computing the inner-product between embeddings.

Problem

• self-play
• theorem proving
• diversity
• conjecturing
• graph structure

Innovation

• self-play
• theorem proving
• graph representation
• diversity maximization
• contrastive learning