verification

This study addresses the limitations of existing SysML verification approaches, which are often tool-dependent and restricted to performance properties, lacking support for automated validation of behavioral and interface requirements. To overcome these shortcomings, this work proposes a tool-agnostic, automated verification workflow driven by SysML test cases, integrating UML Testing Profile and behavioral diagram constructs to enable unified validation of multidimensional attributes—including behavior, timing, and state responses. The methodology was developed through a mixed-methods research strategy combining literature review and stakeholder interviews, and its efficacy was empirically validated across two independent SysML toolchains. The approach not only transcends the constraints of conventional parametric methods but also enables automatic traceability of verification results back to the original model elements.

This paper addresses hyperproperties—higher-order system requirements encompassing information-flow security, knowledge reasoning, and robustness, which span multiple execution traces—by proposing the first unified logical and algorithmic framework covering the entire verification lifecycle. Methodologically, it rigorously characterizes the expressive power and decidability boundaries of classical temporal logics (LTL, CTL, S1S) over hyperproperties; then introduces a novel multi-trace synchronization modeling and quantifier alternation handling mechanism grounded in higher-order temporal logic, constraint solving, and symbolic automata. Key contributions include: (i) a comprehensive taxonomy and complexity-theoretic characterization of hyperproperty logics; (ii) an open-source verification toolchain supporting HyperLTL and HyperCTL*; and (iii) end-to-end support for core verification tasks—including satisfiability checking, model checking, runtime monitoring, and controller synthesis.

This work addresses the high cost of manually writing formal specifications and the limitations of existing large language model (LLM)-based approaches that require white-box access to source code, thereby posing intellectual property and deployment constraints. The authors propose a black-box-driven method that leverages only test code and dynamic execution traces to generate candidate Java Modeling Language (JML) specifications via an LLM. These candidates are locally validated using bounded model checking, and an iterative feedback loop refines them based on verification outcomes. This approach is the first to enable fully automated formal specification generation without any access to the program’s internal structure. Evaluated on the SpecGenBench benchmark, it demonstrates that test-derived information effectively guides specification synthesis, while also highlighting critical challenges in checker compatibility and diagnostic feedback, substantially enhancing industrial applicability.

This work addresses the limited adoption of formal verification, which often requires expert-written annotations such as preconditions, postconditions, and loop invariants. To overcome this barrier, the authors propose a novel approach that leverages large language models (LLMs) in conjunction with assertions from test cases as static oracles to automatically generate Dafny verification annotations from code annotated with natural language comments. The method features an iterative refinement process guided by verifier feedback over multiple rounds and uniquely integrates multi-model LLM collaboration with a closed-loop verifier feedback mechanism. A VS Code plugin was developed to support practical deployment. Evaluated on 110 Dafny programs, the approach achieves a 98.2% annotation correctness rate within at most eight repair iterations. Empirical results highlight that proof-assistant-style annotation remains a key challenge for LLMs, while user feedback on the plugin was notably positive.

This work proposes a novel paradigm that bridges the long-standing divide between testing and formal verification in traditional software validation, enabling them to synergistically enhance both efficiency and quality. Grounded in Design by Contract, the approach leverages the counterexample generation capability of SMT solvers to transform formal verification tools into an integrated engine for automated testing and repair. Within a unified framework, the method simultaneously achieves three key objectives: automatic generation of test cases for faulty programs, construction of regression test suites with full coverage for correct programs, and correctness-guaranteed program repair. This represents the first integration of verification, testing, and repair into a single cohesive methodology.

This study addresses the challenges of applying formal verification to production-grade software, where high modeling costs and consistency risks in fault handling hinder adoption. By integrating runtime execution traces with formal specifications, the authors verify a real-world restaurant point-of-sale (POS) payment workflow and leverage large language models (LLMs) to automatically generate these specifications. Their analysis reveals that the structural form—not the natural language phrasing—of specifications primarily governs LLM-generated correctness, and uncovers a shared “relevant oracle failure” issue between code and simulators. Extending fault models to include crash-recovery, stale reads, and retries, the team conducts simulation-based audits, verifying core protocol correctness, identifying and reproducing seven fault-handling vulnerabilities, and revalidating after fixes. They also expose how deviations in API response structures render recovery paths unreachable—a finding consistently replicated across seven LLMs from two vendors.

This work addresses the challenge that counterexamples generated by formal verification often consist of numerous low-level Boolean variables, rendering them difficult for developers to interpret at the application-domain level. To bridge this gap, the paper proposes a novel hierarchical explanation method that integrates predicate relevance metrics with dependency graph analysis—a first-time fusion of these two techniques—to automatically extract human-readable, domain-oriented explanations from logical formulas. By leveraging formal modeling and a dedicated explanation-generation algorithm, the approach produces concise and semantically clear descriptions of failure causes across multiple case studies. Empirical results demonstrate that the method significantly outperforms existing techniques, offering effective support for fault localization in practical verification tasks.

This work presents the first systematic investigation into the capability of large language models (LLMs) to generate program specifications involving higher-order logical constructs, which are essential for expressing complex verification properties yet remain beyond the reach of existing LLMs that predominantly handle basic syntactic forms. The authors design four syntactic configurations spanning different levels of abstraction and establish a comprehensive evaluation framework to assess a range of representative LLMs on standard verification benchmarks. Experimental results demonstrate that LLMs can effectively produce valid higher-order logical expressions; moreover, integrating logical constructs with base syntax significantly enhances verification efficacy and robustness without substantially increasing verification overhead. The study also reveals distinct advantages of two refinement paradigms in specification generation.

This study addresses the limited adoption of formal methods in electronic warfare systems, which is hindered by steep learning curves, inconsistent terminology, and challenges in integrating with existing development workflows. From the perspective of frontline engineers, the authors empirically evaluate multiple static analysis tools—covering input/output contracts, memory effect analysis, and loop invariants—on safety-critical electronic warfare software, comparing their efficacy against conventional unit testing. The research highlights fundamental differences in reasoning paradigms and tool usage between formal methods and traditional testing, revealing the unique capability of formal techniques to uncover deep-seated security vulnerabilities. Furthermore, the paper identifies concrete directions for improving usability, including reducing manual intervention and enhancing support for library code, thereby facilitating broader practical adoption.