Verification Modulo Tested Library Contracts

2026-04-16

AI Summary

This work addresses the challenge of synthesizing modular specifications for client programs that use complex libraries—specifications that must simultaneously satisfy formal verification requirements and be amenable to empirical testing. To this end, the paper introduces a novel form of specification called *contextual contracts* and develops a counterexample-guided learning framework that integrates constraint solving with black-box testing. The approach leverages an ICE-learning-based generalization engine within a CHC (Constrained Horn Clauses) solver to co-synthesize concise, verifiable, and testable contracts for library methods along with inductive invariants. Experimental evaluation demonstrates that the implemented tool, vmtlc, successfully generates effective contracts meeting both verification and testing criteria on real-world client benchmarks.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Adversarial Learning & Robustness
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We consider the problem of \emph{verification modulo tested library contracts} as a step towards automating the verification of client programs that use complex libraries. We formulate this problem as the synthesis of modular contracts for the library methods used by the client that are adequate to prove the client correct, and that also pass the scrutiny of a testing engine that tests the library against these contracts. We also consider a new form of method contracts called \emph{contextual contracts} that arise in this setting that hold in the context of the client program, and can often be simpler and easier to infer than classical modular contracts. We provide a counterexample-guided learning framework to solve this problem, in which the synthesizer interacts with a constraint solver as well as the testing engine in order to infer adequate modular/contextual method contracts and inductive invariants for the client. The main synthesis engines we use are generalizing CHC solvers that are realized using ICE learning algorithms. We realize this framework in a tool called \vmtlc and show its efficacy on benchmarks where clients call large libraries.

Problem

• verification modulo tested library contracts
• modular contracts
• contextual contracts
• client program verification
• library testing

Innovation

• verification modulo contracts
• contextual contracts
• counterexample-guided synthesis
• ICE learning
• CHC solving