What is the Return on Investment of Digital Engineering for Complex Systems Development? Findings from a Mixed-Methods Study on the Post-production Design Change Process of Navy Assets

2025-10-29

AI Summary

Post-deployment design changes in complex engineering systems frequently cause severe schedule overruns, primarily attributable to inefficient process patterns; moreover, the return on investment (ROI) of Digital Engineering (DE) has long lacked empirical validation. Method: Leveraging real-world U.S. Navy systems engineering data, this study employs a mixed-methods approach to identify four recurrent inefficient process patterns causing schedule delays, quantify their associated scheduling deviations and variability, and formulate a DE-enabled process hypothesis model aligned with the U.S. Department of Defense’s Digital Engineering Strategy. Contribution/Results: This work provides the first quantitative evidence of DE’s ROI in post-production change scenarios: median project cycle time decreases by 50.1%, and schedule standard deviation declines by 41.5%, significantly enhancing execution efficiency and temporal predictability. Furthermore, it demonstrates task-dependent ROI variation, offering empirically grounded guidance for prioritizing DE implementation.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Knowledge Representation and Reasoning: Knowledge Engineering

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Complex engineered systems routinely face schedule and cost overruns, along with poor post-deployment performance. Championed by both INCOSE and the U.S. Department of Defense (DoD), the systems engineering (SE) community has increasingly looked to Digital Engineering (DE) as a potential remedy. Despite this growing advocacy, most of DE's purported benefits remain anecdotal, and its return on investment (ROI) remains poorly understood. This research presents findings from a case study on a Navy SE team responsible for the preliminary design phase of post-production design change projects for Navy assets. Using a mixed-methods approach, we document why complex system sustainment projects are routinely late, where and to what extent schedule slips arise, and how a DE transformation could improve schedule adherence. This study makes three contributions. First, it identifies four archetypical inefficiency modes that drive schedule overruns and explains how these mechanisms unfold in their organizational context. Second, it quantifies the magnitude and variation of schedule slips. Third, it creates a hypothetical digitally transformed version of the current process, aligned with DoD DE policy, and compares it to the current state to estimate potential schedule gains. Our findings suggest that a DE transformation could reduce the median project duration by 50.1% and reduce the standard deviation by 41.5%, leading to faster and more predictable timelines. However, the observed gains are not uniform across task categories. Overall, this study provides initial quantitative evidence of DE's potential ROI and its value in improving the efficiency and predictability of complex system sustainment projects.

Problem

• Quantifying ROI of Digital Engineering for complex Navy system development
• Identifying inefficiency modes causing schedule overruns in sustainment projects
• Estimating potential schedule gains from Digital Engineering transformation

Innovation

• Mixed-methods study analyzes Navy design change processes
• Identifies inefficiency modes causing schedule overruns systematically
• Digital Engineering transformation reduces project duration by half