analyze product metrics

Existing performance measurement frameworks struggle to simultaneously satisfy customizability, interpretability, and mathematical tractability in interdisciplinary contexts. Method: This paper proposes a goal-oriented, customizable metric construction framework featuring a novel “base metric–auxiliary metric” dichotomy. Integrating utility theory and multi-criteria decision analysis, it introduces an uncertainty-aware utility function and establishes a systematic metric decomposition–synthesis workflow. Contributions: (1) It reduces reliance on complex mathematical formalisms, enhancing applicability under resource constraints or high uncertainty; (2) it ensures metric transparency, traceability, and domain adaptability; and (3) it enables quantitative assessment of goal attainment, real-time progress monitoring, and downstream statistical modeling and decision optimization. The framework has been empirically validated across diverse disciplines, demonstrating generality and extensibility.

To address the challenges of fragmented engineering data and inefficient, inconsistent manual reporting in large-scale software development, this paper proposes and implements a centralized engineering productivity analytics framework supporting near-real-time aggregation and visualization. Methodologically, the framework introduces a dual-mode data storage architecture coupled with a precomputation engine, integrated with scheduled data ingestion (cron), proactive alerting, and role-based access control (RBAC). It unifies heterogeneous data from multiple source systems via a dual-schema design and leverages Metabase to enable cross-dimensional visual analytics—spanning development efficiency, software quality, and operational effectiveness. Empirical evaluation demonstrates that the deployed system reduces average weekly manual reporting effort by 20 person-hours, significantly accelerates bottleneck identification latency, and concurrently improves engineering decision responsiveness and platform scalability.

Existing software engineering metrics often fail to effectively support critical development decisions—such as whether refactoring is necessary or whether testing is sufficient—thereby limiting their practical utility. This work addresses this gap by systematically introducing metrological principles (the science of measurement) into the domain of software measurement for the first time. It proposes a metrology-informed approach to metric modeling and evaluation, establishing a rigorous scientific foundation for the design of software metrics. By grounding metric development in established measurement theory, the proposed method substantially enhances the usability, credibility, and decision-support capability of metrics in real-world engineering contexts. This study thus opens a new research direction for software measurement, aligning it more closely with the epistemological standards of empirical science.

Practitioners face significant challenges in effectively transforming customer feedback data into actionable software improvements. Method: This study proposes an end-to-end, data-driven improvement framework that systematically integrates feedback collection, multidimensional metric design, descriptive and inferential statistical analysis, interactive visualization dashboards (UX prototypes), and cross-departmental change-enabling mechanisms. Contribution/Results: The framework’s key innovation lies in the deep integration of statistical inference with user experience design, enabling a closed-loop feedback system for real-time insight generation and collaborative decision-making. Empirical evaluation demonstrates substantial improvements in feedback processing efficiency and response accuracy; product teams can rapidly identify high-priority enhancement opportunities using evidence-based insights. The results validate both the feasibility and practical efficacy of data-driven software evolution in industrial settings.

Business process optimization remains challenging due to fragmented methodologies across process mining, predictive process monitoring, and process-aware recommendation—each operating in isolation without a unified theoretical foundation or integration framework. Method: This paper proposes a closed-loop optimization framework that systematically integrates Alpha algorithm/Inductive Miner for process discovery, LSTM/Transformer for runtime prediction, collaborative filtering/graph neural networks for action recommendation, and explainable AI (XAI) for interpretability—enabling automated bottleneck identification, anomaly forecasting, and prescriptive optimization from event logs. Contribution/Results: We establish the first unified conceptual boundary, evolutionary taxonomy, and synergy paradigm across the three domains; construct a comprehensive classification schema covering 120+ studies; clarify application scopes and standardized evaluation benchmarks; and deliver an industrially actionable methodology selection guide with validated deployment pathways.

This work addresses the unreliability of developer productivity dashboards, which often stems from ad hoc scripts that introduce undetected silent data gaps, eroding organizational trust. To resolve this, we propose a robust ELT pipeline grounded in DAG-based orchestration and the Medallion architecture, decoupling data extraction from transformation to preserve the immutability of raw data. Our approach introduces a state-driven dependency scheduling mechanism and, for the first time, treats metric pipelines as production-grade distributed systems. We emphasize the critical role of immutable raw history in enabling reliable metric redefinition. This methodology significantly enhances data reliability and freshness while effectively eliminating silent failures, thereby restoring organizational confidence in DevOps metrics.

This study addresses a critical limitation of existing DORA metrics, which rely solely on first-order statistics and thus fail to capture the distributional characteristics of software release cadence or distinguish teams with markedly different release regularity. To overcome this, the work introduces second-order statistics into the DORA framework for the first time, proposing a novel Delivery Consistency (DC) metric based on the coefficient of variation of inter-release intervals. It further constructs an eight-prototype Delivery Health Matrix to enable multidimensional diagnosis and targeted intervention for software delivery rhythms across platforms. Validation using real-world data spanning 120 weeks from four platforms—including Jira, GitHub, and Firebase—demonstrates that the approach effectively identifies teams sharing identical DORA ratings yet exhibiting divergent release patterns, uncovering underlying organizational or process constraints common to such teams.

本文提出一种生命周期感知的框架，结合软件质量评估与大语言模型代码优化，以解决科研软件因开发者缺乏软件工程经验导致的质量问题。

This work addresses the challenge of quantifying the academic impact of commercial engineering software such as Ansys Granta, which is hindered by inconsistent citation practices and rapidly growing publication volumes. We propose the first reproducible, semi-automated framework that integrates DOI and citation parsing, expert annotation, and a relational database (Ansys Granta MI Enterprise) to transform heterogeneous usage evidence into a structured knowledge base. As of September 2025, the framework has compiled a multi-source literature repository comprising over 1,100 manually verified records, enabling rapid retrieval, systematic review reproduction, and technology landscape scanning. The resulting knowledge base reveals dominant application domains, key contributing institutions, and integration patterns within CAD/CAE/FEM environments, thereby facilitating systematic tracking and analysis of the long-term technical influence of commercial engineering software.

Frameworks such as SPACE, DevEx, and DORA established that developer productivity is inherently multidimensional, but left practitioners with a practical question: what should we measure, and how should we use it to improve? This paper introduces Engineering Thrive (EngThrive), a measurement and improvement system developed and deployed across Microsoft's engineering organization. EngThrive organizes productivity around three dimensions - Speed, Ease, and Quality - with Thriving as a guardrail to ensure developer wellbeing improves alongside performance. Within each dimension, outcome-oriented North Star metrics are paired with diagnostic submetrics, combining system telemetry with developer surveys to provide both scale and context. We describe the design principles that guide metric selection, including an approach in which well-chosen metrics align "gaming" behavior with genuine improvement. We also outline the data platform, survey program, and dashboard ecosystem required to operationalize this approach in practice, and present case studies demonstrating how outcome-oriented measurement enables sustained, system-level improvements. Finally, we show that EngThrive functions as a general-purpose evaluation language, applicable not only to developer tools and AI, but to organizational policies, work environments, and other factors that shape how developers experience their work. We offer EngThrive as a concrete model for organizations seeking to move beyond measuring activity toward improving outcomes.