formal model specification

This work proposes a human-AI collaborative paradigm for formal software specification that mitigates the traditional barriers to industrial adoption—namely, the notational complexity and high expertise threshold—while preserving the benefits of early error detection and explicit invariants. The approach employs an intermediate language blending natural language with lightweight LaTeX mathematical notation, enabling AI-assisted review, refinement, and code generation. Crucially, it distinguishes between components requiring rigorous formalization and those amenable to flexible treatment. By deeply integrating AI into the specification authoring and verification workflow, this method achieves “correct-by-construction” development in a case study on organizational knowledge growth simulation, significantly reducing costs while ensuring early validation and design correctness.

This study addresses the high cost, error-proneness, and poor maintainability of manually writing Linear Temporal Logic (LTL) specifications—a core bottleneck in formal verification. We systematically survey and evaluate automated LTL specification mining methods. First, we propose a unified, multi-paradigm classification framework—covering constraint solving, neural networks, enumeration-based search, formal language inference, and grammar-guided learning—marking the first such comprehensive taxonomy. Second, we introduce a standardized evaluation methodology assessing scalability, interpretability, and noise robustness across approaches. Our analysis clarifies the applicability boundaries and inherent limitations of each paradigm and yields a practical, industry-oriented selection guide. The work significantly advances the automation level and reliability of LTL specification acquisition, bridging the gap between theoretical mining techniques and real-world verification practice.

Large language models (LLMs) face challenges in formal mathematical verification—including capability coupling, coarse-grained evaluation, and scarcity of high-quality, language-diverse training data. Method: We systematically decouple formal verification into six fine-grained subtasks (e.g., specification translation, proof completion) and construct FM-alpaca, a 18K-sample high-quality instruction-response dataset covering five mainstream formal languages: Coq, Lean4, Dafny, ACSL, and TLA+. Leveraging GPT-4o distillation and supervised fine-tuning (SFT), we propose FM-Bench—the first cross-language, task-decoupled benchmark for formal verification. Contribution/Results: Empirical results show that fine-tuning on formalization data significantly improves formal verification performance (up to 2.9× gain) and positively transfers to mathematical reasoning and programming tasks. Both the model and benchmark are publicly released.

Informal natural language requirements in safety-critical systems impede direct application to formal verification. Method: This paper proposes a semi-automated specification generation approach integrating large language models (LLMs) with domain ontologies, comprising ontology-driven semantic parsing of requirements, LLM-guided instantiation of formal specification templates, and structured reuse of existing specification assets—thereby enhancing verifiability and domain consistency. Contribution/Results: We establish a challenge analysis framework addressing requirement ambiguity, logical incompleteness, and formal mapping deviation. Preliminary validation in aviation and rail transit domains demonstrates a 32% improvement in specification generation accuracy and a 45% reduction in manual correction effort. The work provides a scalable, empirically grounded methodology for trustworthy natural-language-to-formal-specification translation.

Automated verification of interactive console I/O programs in Haskell education remains challenging due to the dynamic, history-dependent nature of student implementations. Method: We propose a lightweight, formal behavioral specification language that uniquely integrates global state and execution history, expressed via regex-like syntax; its trace-based semantics enable probabilistic testing and scalable verification through *sampleable validity*. Contribution/Results: Our system automatically validates student submissions against behavioral specifications and supports pedagogical closed-loop applications—including real-time feedback generation, example solution synthesis, and exercise randomization. Empirical evaluation demonstrates substantial improvements in test coverage and pedagogical adaptability while preserving formal rigor. To our knowledge, this is the first framework for verifying interactive behaviors in functional programming education that simultaneously achieves theoretical soundness and practical deployability.

This work presents the first systematic investigation into the capability of large language models (LLMs) to generate program specifications involving higher-order logical constructs, which are essential for expressing complex verification properties yet remain beyond the reach of existing LLMs that predominantly handle basic syntactic forms. The authors design four syntactic configurations spanning different levels of abstraction and establish a comprehensive evaluation framework to assess a range of representative LLMs on standard verification benchmarks. Experimental results demonstrate that LLMs can effectively produce valid higher-order logical expressions; moreover, integrating logical constructs with base syntax significantly enhances verification efficacy and robustness without substantially increasing verification overhead. The study also reveals distinct advantages of two refinement paradigms in specification generation.

This work addresses the limited adoption of formal verification, which often requires expert-written annotations such as preconditions, postconditions, and loop invariants. To overcome this barrier, the authors propose a novel approach that leverages large language models (LLMs) in conjunction with assertions from test cases as static oracles to automatically generate Dafny verification annotations from code annotated with natural language comments. The method features an iterative refinement process guided by verifier feedback over multiple rounds and uniquely integrates multi-model LLM collaboration with a closed-loop verifier feedback mechanism. A VS Code plugin was developed to support practical deployment. Evaluated on 110 Dafny programs, the approach achieves a 98.2% annotation correctness rate within at most eight repair iterations. Empirical results highlight that proof-assistant-style annotation remains a key challenge for LLMs, while user feedback on the plugin was notably positive.

This work addresses the high cost of manually writing formal specifications and the limitations of existing large language model (LLM)-based approaches that require white-box access to source code, thereby posing intellectual property and deployment constraints. The authors propose a black-box-driven method that leverages only test code and dynamic execution traces to generate candidate Java Modeling Language (JML) specifications via an LLM. These candidates are locally validated using bounded model checking, and an iterative feedback loop refines them based on verification outcomes. This approach is the first to enable fully automated formal specification generation without any access to the program’s internal structure. Evaluated on the SpecGenBench benchmark, it demonstrates that test-derived information effectively guides specification synthesis, while also highlighting critical challenges in checker compatibility and diagnostic feedback, substantially enhancing industrial applicability.

This work addresses the gap in formal methods research, which often relies on simplified examples and lacks accumulated, shared experience in modeling real-world complex systems. Focusing on the formal modeling process itself—not merely verification—the study systematically synthesizes key practical insights and lessons learned from large-scale case studies spanning networks, cyber-physical systems, hardware-software co-design, and biological systems. The project establishes a collection of reusable and comparable modeling exemplars grounded in real systems, thereby compensating for the omission of essential modeling details typically excluded from conventional publications due to space constraints. This repository of detailed models provides a robust foundation for future evaluation of formal methods and advancement of theoretical frameworks.

This study addresses the limitations of existing SysML verification approaches, which are often tool-dependent and restricted to performance properties, lacking support for automated validation of behavioral and interface requirements. To overcome these shortcomings, this work proposes a tool-agnostic, automated verification workflow driven by SysML test cases, integrating UML Testing Profile and behavioral diagram constructs to enable unified validation of multidimensional attributes—including behavior, timing, and state responses. The methodology was developed through a mixed-methods research strategy combining literature review and stakeholder interviews, and its efficacy was empirically validated across two independent SysML toolchains. The approach not only transcends the constraints of conventional parametric methods but also enables automatic traceability of verification results back to the original model elements.