ProofSketcher: Hybrid LLM + Lightweight Proof Checker for Reliable Math/Logic Reasoning

2026-04-07

AI Summary

This work addresses the tendency of large language models (LLMs) to produce mathematically plausible yet logically flawed arguments by proposing an intermediate approach between informal reasoning and full formalization. The method guides LLMs to generate typed, compact proof sketches, which are then automatically expanded by a lightweight trusted kernel into explicit proof obligations. These obligations are expressed in a domain-specific language (DSL) to enable efficient verification. By integrating this structured formal scaffolding with the natural expressiveness of LLM-generated reasoning, the approach preserves human-readable argumentation while introducing rigorous formal guarantees. Empirical results demonstrate that this hybrid strategy significantly enhances both the reliability and correctness of mathematical reasoning performed by LLMs.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The large language models (LLMs) might produce a persuasive argument within mathematical and logical fields, although such argument often includes some minor missteps, including the entire omission of side conditions, invalid inference patterns, or appeals to a lemma that cannot be derived logically out of the context being discussed. These omissions are infamously hard to notice solely out of the text, as even the misconstrued construction still may seem mostly accurate. Conversely, interactive theorem provers like Lean and Coq have rigorous reliability by ensuring that syntactic and semantic statements only accept statements that can pass all the syntactic and semantic steps in the program which is a small trusted kernel of the language type-checks with. Despite the fact that this technique provides strong guarantees, it comes at quite a heavy price: the evidence must be completely formalized, and the evidence user or a auxiliary search program must provide an avalanche of low-level information. This paper presents a hybrid pipeline where an LLM generates a typed proof sketch in a compact DSL and a lightweight trusted kernel expands the sketch into explicit proof obligations.

Problem

• large language models
• mathematical reasoning
• logical reasoning
• formal verification
• proof errors

Innovation

• Hybrid Reasoning
• Proof Sketch
• Lightweight Proof Checker
• Typed DSL
• Formal Verification