quality metrics

This work addresses the problem of implementation drift in evolving distributed systems, where runtime behavior gradually deviates from the original design. To tackle this issue, the paper proposes a design conformance assessment method based on distributed tracing data. It introduces, for the first time in the domain of distributed systems, conformance checking techniques from process mining, leveraging runtime traces collected via the OpenTelemetry standard and automatically comparing them against behavioral models defined at design time to quantify their alignment. The key contribution lies in establishing persistent, monitorable conformance metrics that enable continuous, automated evaluation of deviations between system implementation and design. This approach is readily applicable to modern distributed systems widely adopting OpenTelemetry for observability.

Facing persistent declines in customer satisfaction among small- and medium-sized enterprises (SMEs) in the IT services sector—and intensifying competitive pressure from rivals delivering superior service at lower costs—this study proposes an integrated service quality assessment framework combining Lean Six Sigma and the SERVQUAL model. Methodologically, it systematically unifies the DMAIC methodology, SERVQUAL’s five-dimensional gap analysis, and Six Sigma’s data-driven statistical process control techniques to establish a quantifiable, traceable pathway for service improvement. Empirical validation demonstrates that the framework precisely identifies five root causes of customer dissatisfaction, increases customer satisfaction significantly, and reduces customer acquisition cost by 18.3%. This work bridges a critical methodological gap in service quality management: the absence of a quantitatively rigorous, end-to-end integrated approach—from diagnostic assessment to closed-loop optimization. It offers SME IT service providers a novel quality management paradigm that balances theoretical rigor with practical implementability.

Existing visualization tools for compliance checking lack systematic characterization of analytical tasks, hindering rigorous effectiveness evaluation. This paper introduces the first multidimensional task taxonomy specifically designed for compliance checking, modeling core trace-to-model alignment tasks in process mining along six dimensions: objective, method, constraint type, data characteristics, data target, and cardinality. Crucially, this taxonomy explicitly links the semantic requirements of compliance checking with established visual analytics design principles—thereby bridging the semantic gap between process mining and visual analytics. It provides a reusable theoretical framework to rigorously define visualization purposes, evaluate tool effectiveness, and support co-design of analysis systems. As a result, the interpretability and practical utility of complex compliance analysis outcomes are significantly enhanced.

This study addresses the quality control challenges in additive manufacturing arising from its layer-by-layer fabrication process and high degree of customization. It presents the first systematic adaptation of the Six Sigma DMAIC methodology tailored to this context. By integrating multi-source sensing and measurement data across the entire manufacturing chain—including materials, design, process parameters, and post-processing—the work employs advanced techniques such as deep learning, machine learning, design of experiments, simulation, ontological analysis, and network science to model the complex relationships among design inputs, process variations, and final part quality. The proposed optimization framework enables real-time anomaly detection and simultaneously optimizes lead time and energy consumption, thereby significantly enhancing the quality stability and process controllability of additive manufacturing systems.

Data uncertainties—such as measurement errors, missing values, and erroneous links—undermine the credibility of policy decisions. Method: This paper proposes a decision-stability-oriented sensitivity analysis framework that shifts the analytical focus from parameter deviation to decision robustness. It introduces an interpretable, decision-level sensitivity metric and integrates counterfactual modeling, hypothesis-driven perturbation sampling, decision boundary tracking, and interactive visualization. Contribution/Results: Evaluated on two real-world policy domains—U.S. presidential vote prediction and childhood lead exposure assessment—the framework significantly enhances policymakers’ awareness of analytical robustness, explicitly delineates credible decision intervals, and provides an actionable confidence assessment tool for data-informed policymaking under data imperfections.

Existing software engineering metrics often fail to effectively support critical development decisions—such as whether refactoring is necessary or whether testing is sufficient—thereby limiting their practical utility. This work addresses this gap by systematically introducing metrological principles (the science of measurement) into the domain of software measurement for the first time. It proposes a metrology-informed approach to metric modeling and evaluation, establishing a rigorous scientific foundation for the design of software metrics. By grounding metric development in established measurement theory, the proposed method substantially enhances the usability, credibility, and decision-support capability of metrics in real-world engineering contexts. This study thus opens a new research direction for software measurement, aligning it more closely with the epistemological standards of empirical science.

This work addresses a critical gap in the testing of machine learning (ML) components, where existing approaches predominantly focus on model performance while neglecting system-level quality attributes such as throughput, resource consumption, and robustness—often leading to integration failures. To bridge this gap, the paper proposes the first standalone quality model specifically tailored for ML components. Grounded in the ISO/IEC 25010 quality standard framework and informed by requirements engineering and software quality modeling techniques, the model systematically decouples and structures key quality attributes of ML components, thereby addressing the lack of component-level applicability in ISO/IEC 25059. It provides developers and stakeholders with a unified terminology to prioritize testing efforts. The model’s effectiveness has been validated through user studies and has been integrated into an open-source ML testing tool, enabling practical deployment.

In data-driven economies, organizations lack systematic frameworks for evaluating and managing data value within internal business processes. To address this gap, this study develops a comprehensive data value assessment framework grounded in the Balanced Scorecard’s internal process perspective, integrating three interrelated dimensions: data quality, governance compliance, and operational efficiency. It introduces a novel, multi-layered taxonomy of data value—spanning technological, organizational, and regulatory dependencies—that resolves metric redundancy and establishes cross-dimensional conceptual linkages. Through systematic literature review, theoretical modeling, indicator clustering, and taxonomy design, the research produces a scalable, reusable data value metrics system. This system underpins standardized data valuation models and decision-support systems, offering both a methodological foundation and actionable implementation pathways for cross-sectoral data assetization. (149 words)

This study addresses a critical limitation of existing DORA metrics, which rely solely on first-order statistics and thus fail to capture the distributional characteristics of software release cadence or distinguish teams with markedly different release regularity. To overcome this, the work introduces second-order statistics into the DORA framework for the first time, proposing a novel Delivery Consistency (DC) metric based on the coefficient of variation of inter-release intervals. It further constructs an eight-prototype Delivery Health Matrix to enable multidimensional diagnosis and targeted intervention for software delivery rhythms across platforms. Validation using real-world data spanning 120 weeks from four platforms—including Jira, GitHub, and Firebase—demonstrates that the approach effectively identifies teams sharing identical DORA ratings yet exhibiting divergent release patterns, uncovering underlying organizational or process constraints common to such teams.

This study addresses the lack of systematic evaluation of data quality tools with respect to their measurement capabilities and integration with large language models (LLMs). It presents the first multidimensional assessment framework grounded in real-world enterprise use cases, systematically evaluating six prominent tools—including open-source solutions such as Great Expectations and Deequ, as well as commercial platforms like Informatica and Experian—across dimensions including rule definition, duplicate detection, metric aggregation, and uncertainty handling, along with their LLM integration mechanisms. The findings reveal that commercial tools offer more comprehensive functionality and初步 support for LLM-assisted rule generation, whereas open-source tools provide greater flexibility at the cost of higher implementation effort. Notably, none of the evaluated tools currently enable direct LLM-based data validation. This work provides empirical guidance for selecting data quality tools and advancing their integration with LLMs.