Score
Designing formal data and annotation schemas and constraint specifications that encode implicit assumptions, provide unified output formats (e.g., for CFGs), and define annotation guidelines so labels and toolchains remain consistent and machine-actionable.
Large language models (LLMs) lack verifiability and regulatory alignment when generating compliance-critical artifacts in safety-sensitive domains. Method: We propose Constraint-Guided Verifiable Generation (CVG), a framework featuring a Unified Meta-Model (UMM) for harmonizing heterogeneous regulatory texts; an Integrated Constraint Model (ICM) enabling dual-layer validation—structural (via GBNF/DFA) and semantic (via SHACL/SMT); and a synergistic prefix-safe decoding mechanism coupled with runtime automata and post-generation validators to embed auditable, traceable regulatory evidence chains. Contribution/Results: CVG innovatively integrates machine-verifiable certificates and violation-driven audit-and-repair directly into the generation pipeline. Evaluated on AUTOSAR automotive software and cross-border judicial workflows, CVG achieves 100% structural conformance, reduces manual correction effort by 72%, and seamlessly interoperates with existing Model-Driven Engineering (MDE) toolchains—delivering, for the first time, high-assurance, auditable, end-to-end compliant LLM-generated artifacts.
This work addresses the limited adoption of formal verification, which often requires expert-written annotations such as preconditions, postconditions, and loop invariants. To overcome this barrier, the authors propose a novel approach that leverages large language models (LLMs) in conjunction with assertions from test cases as static oracles to automatically generate Dafny verification annotations from code annotated with natural language comments. The method features an iterative refinement process guided by verifier feedback over multiple rounds and uniquely integrates multi-model LLM collaboration with a closed-loop verifier feedback mechanism. A VS Code plugin was developed to support practical deployment. Evaluated on 110 Dafny programs, the approach achieves a 98.2% annotation correctness rate within at most eight repair iterations. Empirical results highlight that proof-assistant-style annotation remains a key challenge for LLMs, while user feedback on the plugin was notably positive.
In AI-driven connected cyber-physical systems, annotation requirements suffer from ambiguity, misalignment among stakeholders, and unverifiability. Method: This paper proposes the first annotation-requirement–oriented dual-component framework: (1) Annotation Negotiation Cards—ensuring atomicity and stakeholder alignment; and (2) Scenario-based Annotation Specifications—enabling verifiability. Integrating requirements engineering, scenario modeling, collaborative requirement elicitation, and error-attribution mapping analysis, the framework is empirically validated in an autonomous driving perception case study. Results: It covers 18 real-world annotation errors, significantly improving annotation completeness, accuracy, and consistency. This work pioneers the systematic integration of annotation requirements into the requirements engineering discipline, establishing both theoretical foundations and practical tools for trustworthy AI data governance.
This work addresses the challenge of irreproducibility in data analysis scripts, which often stems from implicit assumptions—such as specific package versions, expected data formats, or undocumented manual interventions. The paper proposes a static analysis approach tailored to data analysis workflows that, for the first time, unifies diverse implicit assumptions into inferable constraint models. By leveraging customized program analysis and example-driven modeling, the authors develop a prototype system capable of automatically identifying these hidden assumptions, extracting executable preconditions, and generating verifiable constraints. The resulting framework supports runtime validation and automatic documentation generation, substantially enhancing script executability, reproducibility, and interpretability.
In software design, paradigm-implied semantic expectations—such as data abstraction consistency and feedback-control closed-loop behavior—are often left implicit, leading to design deviations and verification challenges. To address this, we introduce the concept of *design obligations*: explicit, logically formalizable, and verifiable specifications that codify such implicit constraints inherent to design paradigms. Leveraging formal modeling and paradigm semantics analysis, we establish two obligation frameworks—one for data-abstraction-based systems and another for feedback-driven adaptive systems—precisely capturing their core semantic requirements. We demonstrate that common design flaws stem from obligation violations and show how these obligations enable rigorous compliance verification and pedagogical application. This work bridges the semantic gap between design intent and implementation, providing both theoretical foundations and a methodological framework for paradigm-driven design assurance.
This work addresses the problem of global inconsistency in multi-component intelligent agent releases, where local validation passes but cross-component relational integrity fails due to the absence of holistic consistency guarantees. To tackle this, we propose the Schema-SIP Relational Consistency (SIP-RC) framework—the first systematic approach to formally define and mitigate relational inconsistency faults in multi-component deployments. SIP-RC models release packages as graph structures and integrates schema documentation with product contract principles to enable cross-component relational verification. Key mechanisms include declarative–evidential linkage, decision authority scoping, provenance tracking of derived components, and byte-level consistency checks. Preliminary experiments demonstrate the feasibility of the proposed framework, offering a practical and actionable paradigm for ensuring relational consistency in intelligent agent releases.
Large language models (LLMs) excel at function-level code generation but struggle with repository-scale system synthesis due to the ambiguity and unverifiability of natural language prompts, leading to significantly degraded output quality. To address this limitation, this work proposes Structured Specification-Driven Engineering (SSDE), a novel paradigm that, for the first time, leverages structured artifacts as inputs to guide LLMs in generating high-quality, verifiable repository-level code. The feasibility of SSDE is demonstrated through the successful automatic generation of MVC-architected business logic across three real-world software systems. These results highlight SSDE’s potential for large-scale software automation while also uncovering critical challenges and charting promising directions for future research.
This work identifies and formally characterizes a previously undocumented phenomenon—termed “tool-call suppression”—where multiple open-source large language models fail to invoke tools when both tool-calling capabilities and structured output constraints (e.g., JSON Schema) are simultaneously enabled, despite functioning correctly in isolation. The authors propose the “constraint priority inversion” hypothesis, arguing that structured output constraints impose syntactic masking during decoding that inadvertently blocks tokens necessary for tool invocation. To address this, they introduce a transparent, two-stage execution strategy that decouples tool calling from structured generation without requiring model retraining. Experimental results demonstrate that this approach effectively restores tool-calling functionality while preserving strict adherence to output schemas, substantially enhancing agent reliability under joint constraints.
This work addresses the challenge of reliably translating natural language into industrial-grade, deployable SysMLv2 models. The authors propose an iterative generate-check-repair framework that, for the first time, integrates a production-level SysMLv2 conformance checker directly into the generation process as a control mechanism rather than a post-processing step. By combining large language model (LLM) generation with deterministic diagnostic feedback and targeted repair strategies—and terminating only when zero errors remain—the method achieves perfect compliance. Evaluated across 604 test cases derived from 151 prompts and four distinct LLMs, the approach elevates single-pass generation compliance from 51.16% to 100%, enabling robust, direct translation of natural language specifications into engineering-ready SysMLv2 models.