HUNYUANPROVER: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving

2024-12-30

AI Summary

High-quality formalized data is scarce, and search efficiency remains low in interactive theorem proving. Method: We propose HunyuanProver—a Lean 4–specific theorem prover built upon Hunyuan-7B—featuring a scalable, low-cost iterative data synthesis framework for automated formalization and synthetic data generation, coupled with a guided tree search algorithm that emulates “System 2” reasoning to enhance proof discovery. Contribution/Results: HunyuanProver achieves a 68.4% pass rate on miniF2F-test, establishing a new state-of-the-art. It is the first model to formally verify four actual International Mathematical Olympiad (IMO) problems on miniF2F-test. Concurrently, we release a high-quality synthetic dataset comprising 30,000 natural-language problems, their formal statements, and complete Lean 4 proofs—significantly alleviating the formalization data bottleneck.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

We introduce HunyuanProver, an language model finetuned from the Hunyuan 7B for interactive automatic theorem proving with LEAN4. To alleviate the data sparsity issue, we design a scalable framework to iterative synthesize data with low cost. Besides, guided tree search algorithms are designed to enable effective ``system 2 thinking`` of the prover. HunyuanProver achieves state-of-the-art (SOTA) performances on major benchmarks. Specifically, it achieves a pass of 68.4% on the miniF2F-test compared to 65.9%, the current SOTA results. It proves 4 IMO statements (imo_1960_p2, imo_1962_p2}, imo_1964_p2 and imo_1983_p6) in miniF2F-test. To benefit the community, we will open-source a dataset of 30k synthesized instances, where each instance contains the original question in natural language, the converted statement by autoformalization, and the proof by HunyuanProver.

Problem

• Automated Theorem Proving
• Mathematical Proof
• Data Scarcity

Innovation

• HunyuanProver
• automated theorem proving
• efficient search algorithm