Assisting Mathematical Formalization with A Learning-based Premise Retriever

2025-01-21

AI Summary

Novices in formal mathematics face challenges in selecting appropriate premises for proof construction, compounded by scarce training data for premise selection. Method: We propose the first contrastive learning framework for premise retrieval tailored to proof states. Our approach introduces a domain-specific tokenizer and fine-grained semantic similarity computation; integrates a BERT-based encoder with vector-space retrieval; and incorporates a lightweight re-ranking module to enhance precision. Contribution/Results: We adapt contrastive learning to formal proof-state modeling—enabling accurate, efficient retrieval of theorems from Mathlib. Experiments demonstrate substantial improvements in retrieval accuracy over state-of-the-art baselines, alongside reduced computational overhead. We release an open-source, end-to-end search tool that significantly improves accessibility and efficiency in formal theorem proving.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Premise selection is a crucial yet challenging step in mathematical formalization, especially for users with limited experience. Due to the lack of available formalization projects, existing approaches that leverage language models often suffer from data scarcity. In this work, we introduce an innovative method for training a premise retriever to support the formalization of mathematics. Our approach employs a BERT model to embed proof states and premises into a shared latent space. The retrieval model is trained within a contrastive learning framework and incorporates a domain-specific tokenizer along with a fine-grained similarity computation method. Experimental results show that our model is highly competitive compared to existing baselines, achieving strong performance while requiring fewer computational resources. Performance is further enhanced through the integration of a re-ranking module. To streamline the formalization process, we will release a search engine that enables users to query Mathlib theorems directly using proof states, significantly improving accessibility and efficiency. Codes are available at https://github.com/ruc-ai4math/Premise-Retrieval.

Problem

• Theorem Selection
• Data Scarcity
• Novice Difficulty

Innovation

• BERT model
• theorem retrieval
• re-ranking step