PriorProof: A Point-in-Time Measure of Technique Novelty for Formal Proofs

2026-07-18

AI Summary

This work proposes a fully automated, interpretable novelty metric for Lean proof paths that requires no human annotations or technical ontologies, objectively quantifying their deviation from historical mathematical knowledge. The method analyzes the dependency footprint of proof terms, constructs a temporally anchored hierarchical smoothing prior using early Mathlib snapshots, and computes a weighted surprisal score to assess novelty. Experimental evaluation demonstrates strong alignment with expert judgment, achieving 69.7% agreement on 76 blind-rated pairs and 84.2% accuracy in the subset with maximal score separation, thereby validating both the reliability and endpoint calibration of the proposed measure.

Abstract

Mathematicians distinguish proofs that explain, simplify, or introduce a nonstandard route, but these judgments are difficult to operationalize. We study a deliberately narrower construct: time-relative proof-route nonstandardness in formal mathematics. For a Lean theorem, PriorProof extracts the dependency footprint of its elaborated proof term and scores the weighted surprisal of that footprint under a retrieval-conditioned, hierarchically smoothed prior built only from an earlier quarterly snapshot of Mathlib. The method requires no hand-built technique ontology and no human labels: statement retrieval is learned from proof-derived contrastive pairs, while the scored object is read mechanically from proof terms. In a blinded topology study, 100 presentations collapse to 76 distinct underlying pairs: 12 canonical contrasts shown three times for consistency screening and 64 distinct stratified pairs. Against the majority of three retained domain raters, PriorProof agrees on 53/76 pairs (69.7%, Wilson 95% CI 58.7-78.9%), including 11/12 canonical pairs (91.7%, 64.6-98.5%) and 42/64 stratified pairs (65.6%, 53.4-76.1%). Score-gap quartiles are nonmonotone after repeat collapse; the endpoints are 12/19 (63.2%, 41.0-80.9%) in the smallest-gap bin and 16/19 (84.2%, 62.4-94.5%) in the largest, supporting an endpoint-calibration tendency rather than a resolved staircase. The best language-model condition agrees on 60/76 pairs (78.9%, 68.5-86.6%); on paired outcomes, PriorProof alone is correct on 8 pairs and the model alone on 15 (exact two-sided McNemar p = 0.210), so the difference is not established at this sample size. We therefore present PriorProof not as a replacement for expert or model judgment, but as a decomposable, time-anchored signal whose score gap provides an interpretable reliability indicator.

Problem

• proof novelty
• formal proofs
• time-relative nonstandardness
• dependency footprint
• Mathlib

Innovation

• proof novelty
• formal mathematics
• dependency footprint
• time-relative surprisal
• retrieval-conditioned prior