data validation

This study addresses the limitations of existing SysML verification approaches, which are often tool-dependent and restricted to performance properties, lacking support for automated validation of behavioral and interface requirements. To overcome these shortcomings, this work proposes a tool-agnostic, automated verification workflow driven by SysML test cases, integrating UML Testing Profile and behavioral diagram constructs to enable unified validation of multidimensional attributes—including behavior, timing, and state responses. The methodology was developed through a mixed-methods research strategy combining literature review and stakeholder interviews, and its efficacy was empirically validated across two independent SysML toolchains. The approach not only transcends the constraints of conventional parametric methods but also enables automatic traceability of verification results back to the original model elements.

Ad hoc SQL development lacks engineering rigor, leading to data silos, logical redundancy, and ineffective data governance. Method: This paper proposes a DataOps-driven CI/CD framework for analytical SQL warehouses, featuring a novel five-stage automated pipeline—Lint, Optimize, Parse, Validate, Observe—that embeds quality assurance and enables end-to-end lifecycle governance. Contribution/Results: We introduce the DataOps Controls Scorecard and a requirements traceability matrix, explicitly mapping 12 governance criteria to CI/CD stages to ensure control completeness and scalability. The framework integrates Agile, Lean, and DevOps principles with static analysis, syntactic parsing, optimization recommendations, validation testing, and observability. Empirical evaluation demonstrates significant improvements in data quality, development transparency, and cross-functional collaboration, providing a sustainable, production-ready pathway for large-scale analytical systems.

In industrial settings, limited production data severely compromises the fidelity of test data for SQL generation services (e.g., NL2SQL), hindering simultaneous preservation of structural integrity and semantic coherence. Method: This paper proposes an LLM-driven high-fidelity test data generation method, integrating Gemini with schema-aware preprocessing, SQL-semantic alignment postprocessing, and constraint-guided sampling—supporting complex patterns including nested columns, multi-table JOINs, aggregations, and deep subqueries. Contribution/Results: The method jointly optimizes semantic consistency, syntactic correctness, and structural fidelity, significantly improving test coverage and defect detection rates. Evaluated on Google’s real-world NL2SQL workloads, it generates high-quality mock data out-of-the-box, effectively addressing the semantic incoherence prevalent in existing approaches under large-scale, complex database schemas.

This study addresses the lack of effective validation methods for semi-formal blueprints in early-stage software product line engineering, which often leads to undetected structural and constraint-related errors in feature models. For the first time, it systematically evaluates the capability of large language models (LLMs) in feature model analysis tasks by leveraging twelve state-of-the-art LLMs and sixteen standard analytical operations that integrate structural parsing with constraint reasoning. Performance is benchmarked against the solver-based tool FLAMA. Results demonstrate that reasoning-optimized models—such as Grok 4 Fast Reasoning and Gemini 2.5 Pro—achieve average accuracies of 88–89%, approaching the performance of formal solvers. These findings substantiate the feasibility and practical potential of LLMs as lightweight, early-stage validation tools for feature model verification.

This work addresses the challenge that domain experts face in translating natural language descriptions of data quality requirements into executable analyses, a process often hindered by reliance on data engineers, resulting in inefficiency and high technical barriers. To overcome this, the paper proposes a no-code, model-driven pipeline that leverages a QPM metamodel to define domain-specific quality analysis templates. Coupled with the Constrainify toolchain, it automatically transforms natural language requirements into executable and reusable analytical logic. By integrating model-driven engineering, metamodeling, and no-code web technologies, the approach significantly reduces dependency on technical expertise, enabling efficient, reproducible, and semantically aligned data quality assessments. This advancement enhances both the accessibility and automation of data quality analysis for non-technical domain practitioners.

This work addresses the limited reliability of structured security artifacts—such as KQL queries and MITRE ATT&CK mappings—generated by large language models, which often fall short of production-grade requirements. To bridge this gap, the authors propose a lightweight verification framework that shifts the focus of quality assurance from generation to validation. The core innovations include a hybrid testing strategy integrating test-driven generation, deterministic program verification, and semantic evaluation by large language models, alongside an interpretable judging mechanism distilled from expert decision distributions. Deployed in Microsoft Sentinel’s production environment, the framework significantly enhances the reliability of three critical types of security artifacts, establishing professional-grade and scalable validation standards.

This study addresses the lack of traceable, structured linkage between high-level requirements and low-level automated testing in AI-enabled cyber-physical systems, which hinders compliance with regulatory demands for verifiable evidence. To bridge this gap, the paper introduces VNVSpec, a novel framework that enables end-to-end automated traceability and closed-loop verification from high-level engineering requirements to test cases. VNVSpec employs machine-readable verification and validation (V&V) specifications to support requirement ingestion, quality checks, metric-driven decomposition, test result association, and generation of audit-ready reports, all integrated into a continuous integration pipeline. Empirical evaluation demonstrates that the approach verifies 36 requirements against 449 tests in linear time, scales to tens of thousands of artifacts, and is fully reproducible through open-sourced code, test suites, and benchmark scripts.

This work addresses the lack of systematic validation for unit tests generated by foundation models, which hinders reliable assessment of their correctness, utility, and maintainability. To bridge this gap, we introduce TestMap—an open-source infrastructure tailored for C#/.NET projects—that establishes an evidence-centered framework for automated test generation across its entire lifecycle, encompassing mapping, execution, repair, evaluation, and experiment tracking. By integrating repository analysis, source-test mapping, coverage and mutation testing, static analysis, test smell detection, and model-guided generation, TestMap enables observable, reproducible, and comparable experimentation across diverse models, prompts, and strategies. The framework further uncovers limitations of current models, missing contextual information, repair overhead, and latent defects in the system under test, thereby providing an empirical foundation for trustworthy test generation.

This work addresses the challenge of reliably translating natural language into industrial-grade, deployable SysMLv2 models. The authors propose an iterative generate-check-repair framework that, for the first time, integrates a production-level SysMLv2 conformance checker directly into the generation process as a control mechanism rather than a post-processing step. By combining large language model (LLM) generation with deterministic diagnostic feedback and targeted repair strategies—and terminating only when zero errors remain—the method achieves perfect compliance. Evaluated across 604 test cases derived from 151 prompts and four distinct LLMs, the approach elevates single-pass generation compliance from 51.16% to 100%, enabling robust, direct translation of natural language specifications into engineering-ready SysMLv2 models.

This work addresses the problem of global inconsistency in multi-component intelligent agent releases, where local validation passes but cross-component relational integrity fails due to the absence of holistic consistency guarantees. To tackle this, we propose the Schema-SIP Relational Consistency (SIP-RC) framework—the first systematic approach to formally define and mitigate relational inconsistency faults in multi-component deployments. SIP-RC models release packages as graph structures and integrates schema documentation with product contract principles to enable cross-component relational verification. Key mechanisms include declarative–evidential linkage, decision authority scoping, provenance tracking of derived components, and byte-level consistency checks. Preliminary experiments demonstrate the feasibility of the proposed framework, offering a practical and actionable paradigm for ensuring relational consistency in intelligent agent releases.