code specification extraction

This work proposes a human-AI collaborative paradigm for formal software specification that mitigates the traditional barriers to industrial adoption—namely, the notational complexity and high expertise threshold—while preserving the benefits of early error detection and explicit invariants. The approach employs an intermediate language blending natural language with lightweight LaTeX mathematical notation, enabling AI-assisted review, refinement, and code generation. Crucially, it distinguishes between components requiring rigorous formalization and those amenable to flexible treatment. By deeply integrating AI into the specification authoring and verification workflow, this method achieves “correct-by-construction” development in a case study on organizational knowledge growth simulation, significantly reducing costs while ensuring early validation and design correctness.

This work addresses the high cost of manually writing formal specifications and the limitations of existing large language model (LLM)-based approaches that require white-box access to source code, thereby posing intellectual property and deployment constraints. The authors propose a black-box-driven method that leverages only test code and dynamic execution traces to generate candidate Java Modeling Language (JML) specifications via an LLM. These candidates are locally validated using bounded model checking, and an iterative feedback loop refines them based on verification outcomes. This approach is the first to enable fully automated formal specification generation without any access to the program’s internal structure. Evaluated on the SpecGenBench benchmark, it demonstrates that test-derived information effectively guides specification synthesis, while also highlighting critical challenges in checker compatibility and diagnostic feedback, substantially enhancing industrial applicability.

Formal program specifications are notoriously difficult, error-prone, and inefficient to write manually. To address this, we propose a two-stage LLM-driven approach: dialogue-guided specification synthesis followed by mutation-based verification. First, multi-turn dialogues model complex semantic requirements; second, four mutation operators—insertion, replacement, deletion, and reordering—enable verifiability-driven selection, eliminating reliance on rigid templates or syntactic grammars. Our method integrates code understanding, prompt engineering, and heuristic verifiability assessment. Evaluated on SV-COMP and a custom Java benchmark comprising 385 programs, it generates 279 verifiable specifications. These achieve significantly higher completeness and accuracy than pure-LLM baselines and classical tools (e.g., Houdini, Daikon). To our knowledge, this is the first approach to achieve both high coverage and formal verifiability in fully automated specification generation.

Automatically verifying C programs generated by large language models (LLMs) remains challenging due to their syntactic and semantic irregularities, which hinder formal verification. Method: This paper proposes SynVer—a novel framework that tightly integrates LLM-based program synthesis with formal verification. SynVer introduces verifiability-aware biasing mechanisms operating at both syntactic and semantic levels to guide LLMs toward generating verification-friendly code. It further incorporates separation logic (SL) specifications and the Verified Software Toolchain (VST) to enable end-to-end, fully automated verification—from specification to C implementation to machine-checked safety proofs. Results: Evaluated on diverse benchmarks covering basic coding tasks, SL assertions, and API specifications, SynVer significantly improves the automatic verification success rate of LLM-generated C programs. Empirical results demonstrate its scalability, robustness, and effectiveness in bridging the gap between neural code generation and rigorous formal assurance.

Automated verification of interactive console I/O programs in Haskell education remains challenging due to the dynamic, history-dependent nature of student implementations. Method: We propose a lightweight, formal behavioral specification language that uniquely integrates global state and execution history, expressed via regex-like syntax; its trace-based semantics enable probabilistic testing and scalable verification through *sampleable validity*. Contribution/Results: Our system automatically validates student submissions against behavioral specifications and supports pedagogical closed-loop applications—including real-time feedback generation, example solution synthesis, and exercise randomization. Empirical evaluation demonstrates substantial improvements in test coverage and pedagogical adaptability while preserving formal rigor. To our knowledge, this is the first framework for verifying interactive behaviors in functional programming education that simultaneously achieves theoretical soundness and practical deployability.

Existing large code models struggle to generate executable intermediate formal specifications, limiting precise verification and repair of program behavioral errors. This work proposes SpecCoder, a novel framework that focuses on generating executable inline assertions at critical program locations, thereby transforming static annotations into verifiable evidence. SpecCoder employs verification-guided training, fine-tuning the Qwen2.5-Coder series models using correct programs, behavioral mutants, and multi-round specification refinement trajectories. Evaluated on the HumanExec benchmark, SpecCoder substantially improves the correctness (+55.8%), completeness (+358.1%), and assertion validity (+26.6%) of inline specifications, significantly enhancing program verification and repair capabilities.

This work addresses the challenge of statically verifying semantic consistency between natural language business requirements and their code implementations. It proposes a two-stage, runtime-free approach: first leveraging large language models to extract structured rules from requirements while identifying ambiguous or contradictory statements, and then performing static code auditing based on this intermediate representation. By integrating natural language processing with static analysis, the method mitigates hallucination and context loss in large models through rule structuring, enabling requirement-aware early validation. Evaluated on an automotive cybersecurity case study, the approach successfully detects semantic deviations, offers a novel solution to the test oracle problem, and significantly enhances left-shifted verification capabilities.

Large language models (LLMs) excel at function-level code generation but struggle with repository-scale system synthesis due to the ambiguity and unverifiability of natural language prompts, leading to significantly degraded output quality. To address this limitation, this work proposes Structured Specification-Driven Engineering (SSDE), a novel paradigm that, for the first time, leverages structured artifacts as inputs to guide LLMs in generating high-quality, verifiable repository-level code. The feasibility of SSDE is demonstrated through the successful automatic generation of MVC-architected business logic across three real-world software systems. These results highlight SSDE’s potential for large-scale software automation while also uncovering critical challenges and charting promising directions for future research.

This study addresses the challenge of automatically detecting software design patterns in source code to support architectural understanding and quality assessment. It presents the first systematic evaluation of four large language models—including NextCoder and Gemma 3—as well as two ensemble strategies combining three models, for recognizing five classic design patterns: Singleton, Adapter, Bridge, Composite, and Decorator. The work investigates the impact of three input modalities—raw source code, PlantUML diagrams, and textual descriptions—on detection performance. Experimental results demonstrate that NextCoder and Gemma 3 achieve the highest accuracy among individual models, while ensemble approaches further enhance performance, thereby confirming the effectiveness and potential of large language models in design pattern recognition tasks.