Lemmanaid: Neuro-Symbolic Lemma Conjecturing

2025-04-07

AI Summary

Automated lemma conjecturing in formal mathematics faces a fundamental trade-off among novelty, utility, and verifiability. Method: This paper introduces the first neuro-symbolic framework: a fine-tuned large language model (LLM) generates generalizable lemma templates, which are then instantiated, constraint-solved, and formally verified by the Isabelle/HOL symbolic engine. The framework establishes a novel “templated division of labor” paradigm, synergistically combining the LLM’s creative generalization with symbolic reasoning’s logical precision. Contribution/Results: Evaluated across diverse Isabelle proof corpora, our approach substantially outperforms both end-to-end LLMs and traditional symbolic lemma generators. It successfully produces domain-diverse, formally verified, and reusable lemmas—enhancing proof automation, lowering barriers to formalization, and accelerating theory development in interactive theorem proving.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Machine Learning: Neuro-Symbolic Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Automatically conjecturing useful, interesting and novel lemmas would greatly improve automated reasoning tools and lower the bar for formalizing mathematics in proof assistants. It is however a very challenging task for both neural and symbolic approaches. We present the first steps towards a practical neuro-symbolic lemma conjecturing tool, Lemmanaid, that combines Large Language Models (LLMs) and symbolic methods, and evaluate it on proof libraries for the Isabelle proof assistant. We train an LLM to generate lemma templates that describe the shape of a lemma, and use symbolic methods to fill in the details. We compare Lemmanaid against an LLM trained to generate complete lemma statements as well as previous fully symbolic conjecturing methods. Our results indicate that neural and symbolic techniques are complementary. By leveraging the best of both symbolic and neural methods we can generate useful lemmas for a wide range of input domains, facilitating computer-assisted theory development and formalization.

Problem

• Automated lemma generation for proof assistants
• Combining neural and symbolic methods effectively
• Enhancing computer-assisted theory development

Innovation

• Combines LLMs and symbolic methods
• Generates lemma templates with LLMs
• Fills details using symbolic techniques