🤖 AI Summary
Machine learning systems lack effective, quantifiable methods to assess how architectural design patterns impact scalability, performance, and cost—leading to subjective, evidence-deficient architecture selection. Method: This paper introduces the first quantitative architectural evaluation framework tailored for ML systems, specifically targeting CPU-based inference. It explicitly models the mappings between architectural patterns and key quality attributes: latency, throughput, and resource overhead. The framework integrates metric-driven modeling, lightweight observability analysis, and standardized benchmarking procedures. Contribution/Results: Evaluated across multiple case studies, the framework enables objective, quantitative ranking of architectural patterns—achieving up to 2.3× higher inference throughput and an average 37% improvement in CPU utilization. It further supports cost-optimized decision-making in production environments. Its core contribution is the establishment of the first rigorous, quantification-oriented paradigm for evaluating ML architectural patterns.
📝 Abstract
Contemporary intelligent systems incorporate software components, including machine learning components. As they grow in complexity and data volume such machine learning systems face unique quality challenges like scalability and performance. To overcome them, engineers may often use specific architectural patterns, however their impact on ML systems is difficult to quantify. The effect of software architecture on traditional systems is well studied, however more work is needed in the area of machine learning systems. This study proposes a framework for quantitative assessment of architectural patterns in ML systems, focusing on scalability and performance metrics for cost-effective CPU-based inference. We integrate these metrics into a systematic evaluation process for selection of architectural patterns and demonstrate its application through a case study. The approach shown in the paper should enable software architects to objectively analyze and select optimal patterns, addressing key challenges in ML system design.