Neuroforger: certified violation witnesses for smart contracts verification via LLMs

2026-05-29

AI Summary

This work addresses the reliability challenges of large language models (LLMs) in smart contract verification, which stem from the ambiguity of natural language specifications and the lack of correctness guarantees. To overcome these limitations, the paper proposes a novel verification pipeline that integrates LLMs with formal methods. It introduces an extended Solidity specification language featuring abstract types—a first-time application of abstract types to smart contract specifications—and constructs a collaborative verification framework where LLMs interact with type checking, abstract interpretation, and concrete execution. This framework automatically generates executable and certifiable proof-of-concept counterexamples that violate specified properties. Experimental evaluation on standard smart contract verification benchmarks demonstrates the effectiveness and practical potential of the proposed approach.

Abstract

Recent large language models (LLMs) incorporate reasoning capabilities that allow them to perform well in predicting whether a smart contract respects a certain property, suggesting a complementary approach to traditional formal-methods-based techniques for smart contract verification. However, the application of LLMs in such context has two major issues: 1) properties expressed in natural language are intrinsically ambiguous, and 2) answers returned by LLMs have no guarantee of correctness. In this paper, we address both issues simultaneously by: 1) introducing a new formal specification language that extends Solidity with abstract types, and 2) designing a workflow that combines LLMs with type checking and concrete execution to generate and validate violation witnesses (i.e., counterexamples). The key idea is to represent a specification as a Solidity test with (existentially quantified) variables of abstract type; finding an instantiation of these variables to concrete values (of the correct type) concretizes the test into an executable counterexample (PoC) for the target property. We implemented our procedure in the tool Neuroforger, experimentally evaluating it on a smart-contract verification dataset drawn from literature, obtaining promising results that demonstrate its potential applicability in the wild.

Problem

• smart contract verification
• large language models
• formal specification
• violation witnesses
• correctness guarantee

Innovation

• smart contract verification
• large language models
• formal specification
• violation witnesses
• abstract types