HarnessLLM: Rust Verification Harness Generation with Large Language Models

2026-07-24

AI Summary

This work addresses the persistent memory safety risks in Rust arising from unsafe code and panics, which existing formal verification approaches struggle to mitigate due to their reliance on manually crafted test harnesses—a process that is both labor-intensive and costly. To overcome this limitation, the paper presents the first application of large language models to automated memory safety verification in real-world Rust projects. It introduces a novel harness generation method that integrates static dependency analysis, nondeterministic parameter synthesis, and iterative program synthesis. By extracting invocation contexts from existing tests and refining outputs through a feedback-driven mechanism for type and function stubbing, the approach achieves 94.66% precision in recovering 294 call sites across nine real-world projects, generating each harness in an average of 145 seconds—substantially outperforming baseline techniques—and successfully uncovers six previously unknown memory safety vulnerabilities.

Abstract

Rust's ownership model and type system offer strong memory safety guarantees, but unsafe code and runtime panics still present significant risks. Formal verification is essential to ensure memory safety, but developing verification harnesses remains a challenging and manual task. Although large language models (LLMs) have shown strong performance in various code analysis tasks, directly applying them to harness generation often results in inaccurate API invocations, inefficient nondeterministic data generation, and fabricated fixes. In this paper, we present HarnessLLM, an automated workflow that leverages LLMs to generate verification harnesses for Rust code directly from existing test suites. HarnessLLM automatically extracts calling scenarios from test cases, generates nondeterministic arguments based on dependency analysis, and incrementally synthesizes harnesses. It then iteratively refines the harnesses, preserving critical code regions and reporting fabricated types or functions to LLMs for correction. In our evaluation on 9 real-world Rust codebases, HarnessLLM extracted 294 calling scenarios from 494 test cases with 94.66% precision and generated harnesses for all scenarios in an average of 145 seconds each. It outperformed the existing approach, Autoharness, which succeeded on only 41% of those scenarios. Finally, 6 real-world memory safety bugs were detected using the generated harnesses, demonstrating the practical utility of our approach in verification. To our knowledge, this is the first work to use LLMs for generating harnesses aimed at memory safety verification in real-world Rust projects.

Problem

• Rust verification
• harness generation
• memory safety
• large language models
• formal verification

Innovation

• Rust verification
• LLM-based harness generation
• memory safety
• nondeterministic input synthesis
• iterative refinement