scaling analysis

Quantifying how input uncertainty propagates to model outputs remains a fundamental challenge in computational modeling. Method: This study systematically reviews and empirically compares prominent global and local sensitivity analysis (SA) techniques—including Sobol’, FAST, Morris screening, and local derivative-based methods—implemented via standard software packages, supporting both probabilistic modeling and distribution-free settings. Contribution/Results: We propose a practical decision framework that guides method selection based on problem characteristics, analytical objectives, and resource constraints—rejecting the notion of a universally “optimal” SA method and thereby addressing a critical gap in methodological implementation guidance. A reusable, open-source toolkit is developed to enhance the reliability and interpretability of uncertainty attribution. The framework and tools have been validated across multiple engineering and policy modeling applications, demonstrating robustness and scalability in real-world contexts.

Metastable failures are rare yet high-risk failure modes in cloud systems, triggered by transient load spikes and persisting as prolonged performance degradation even after stress subsides. This paper addresses request-response server systems by proposing a continuous-time Markov chain (CTMC)-based modeling framework. It introduces the first formal definition of metastability via escape probability and establishes a quantitative relationship between metastability and the spectral gap of the CTMC’s dominant eigenvalues—enabling computationally tractable recovery-time prediction and visual identification of metastability-prone parameter configurations. The methodology integrates domain-specific language modeling, data-driven calibration, and combined qualitative/quantitative analysis. The developed tool detects diverse real-world metastable phenomena within milliseconds. Experimental validation confirms the critical phenomenon: as system parameters approach the metastable regime, recovery time grows exponentially.

Accurately modeling and quantitatively evaluating performance bottlenecks in real-world systems remains challenging. Method: This paper proposes a theory-driven, practice-oriented, progressive performance modeling framework that integrates queuing theory, Markov models, load-testing-based modeling, and system simulation. It employs a three-tiered problem design—foundational modeling → dynamic workload analysis → industrial-scale system simulation—to enable capability transfer from classroom training to complex system analysis. Contribution/Results: The framework innovatively couples quantitative modeling techniques with hierarchical pedagogical practices, establishing a scalable, verifiable, integrated teaching–practice ecosystem for performance evaluation. Experimental results demonstrate significant improvements in learners’ modeling accuracy and solution efficiency for large-scale system performance problems; the framework has been successfully deployed in multiple industrial system performance optimization scenarios.

Current cyber-physical systems (CPS) in vehicular environments lack quantitative, experimentally grounded methods for assessing network resilience. Method: This study constructs an experimental testbed replicating real-world truck operational conditions and conducts multiple rounds of malware injection attacks, simultaneously collecting network- and physical-layer data on resistance and recovery behaviors. Contribution/Results: We introduce the novel concept of “bonware” to holistically characterize both cybersecurity defense capability and physical resilience, formalized via an analytically tractable mathematical model. We further define and extract experimentally identifiable, quantitative resilience metrics—termed elastic features—for the first time. Sensitivity analysis confirms these metrics exhibit significant discriminability with respect to attack intensity, defensive strategies, and physical redundancy. This work bridges a critical gap by advancing vehicular CPS resilience from qualitative description to quantifiable, comparable, and optimizable measurement.

Existing power grid resilience research remains largely conceptual or focuses on isolated components, lacking a system-level, quantifiable definition and assessment framework. Method: Leveraging 15-minute-resolution customer outage time-series data and high-resolution meteorological records, we develop a spatiotemporal statistical model incorporating resilience sensitivity simulation and outage propagation dynamics inference. Contribution/Results: We propose the first system-level, empirically measurable definition of grid resilience. The model uncovers cumulative outage effects under extreme weather, inter-regional outage propagation mechanisms, and systemic response patterns. It identifies critical reinforcement nodes that reduce customer outage magnitude by nearly 50%. Validated across three major U.S. East Coast utility service territories, the model achieves high accuracy in forecasting outage progression—enabling actionable support for real-time dispatch decisions and emergency response.

This work addresses the poor robustness and low predictive accuracy of machine learning model scaling laws. We propose a systematic, reproducible framework for scaling law modeling and evaluation. Based on large-scale empirical analysis across 485 pretrained models on downstream tasks, we (i) demonstrate for the first time that leveraging intermediate training checkpoints significantly improves fitting accuracy; (ii) find that parameter transfer between isomorphic models outperforms cross-size extrapolation; and (iii) verify that averaging estimates from multiple small models trained with different random seeds is more robust than relying on a single large model. We release the first open-source, multidimensional scaling law benchmark dataset, integrating log-log linear regression, training trajectory analysis, and statistical robustness assessment. Our framework reduces average prediction error by 37% across diverse architectures, providing efficient and reliable quantitative guidance for key pretraining decisions—including optimizer selection, dataset curation, and architectural design.

This study addresses how to reliably predict the upper-bound performance of language models on downstream tasks based on pretraining compute budgets and assess the temporal stability of this mapping. Leveraging large-scale empirical observations, the work employs smooth quantile regression to model the monotonic, saturating S-shaped relationship between pretraining FLOPs and high-percentile downstream performance, establishing the first stable and predictable performance frontier. Key contributions include uncovering the continually advancing frontier of mathematical reasoning capabilities, proposing an efficient algorithm that approximates the full performance frontier using only 20% of the evaluation budget, and releasing the Proteus-2k benchmark dataset. Empirical validation demonstrates strong temporal stability of capability frontiers across most tasks—except mathematical reasoning—substantially reducing evaluation costs.

This study investigates the co-optimization of model scale, dataset size, and numerical precision in low-precision training to balance performance and computational cost. Leveraging a high-dimensional sketching-based linear regression framework, the work models quantization error and analyzes theoretical scaling laws to reveal a fundamental distinction between multiplicative and additive quantization: the former preserves the effective capacity of full-precision models, whereas the latter substantially diminishes it. Theoretical analysis characterizes the intricate coupling among model size, data volume, and precision, and extensive experiments confirm markedly different scaling behaviors between the two quantization paradigms. These findings provide a principled foundation and practical design guidelines for efficient low-precision training.

Traditional scaling law estimation suffers from high computational costs due to the absence of efficient budget allocation strategies. This work proposes a novel approach that, for the first time, integrates surrogate-guided pruning into scaling law modeling by combining the Successive Halving algorithm with both parametric and non-parametric surrogate models. This integration enables proactive allocation of computational resources and efficient construction of loss-compute Pareto frontiers. The method substantially improves resource utilization efficiency, achieving relative performance gains of up to 2.84% on real datasets and 5.47% on synthetic datasets, while reducing computational costs by as much as 98.7%.

Existing data center simulators often rely on single-model approaches, limiting their ability to accurately assess both performance and carbon emissions while lacking adaptability to diverse operational scenarios. This work proposes the M3SA framework, which establishes a multi-model cooperative simulation architecture by integrating discrete-event simulation with meta-model ensembles. Implemented within the open-source OpenDC platform, M3SA enables joint dynamic evaluation of performance and climate impact. The framework supports cross-workload and multi-timescale “what-if” analyses, accurately predicts operational phenomena such as failures, and facilitates CO₂-aware scheduling and other optimization strategies. Experimental results reproduce and extend prior studies, and the implementation is publicly released.

This work investigates how model width and training sample size jointly influence the generalization performance of finite-width, two-layer quadratic neural networks with ℓ² regularization under structured, finite-sample data regimes. Leveraging spectral analysis and finite-sample generalization theory, the study establishes—for the first time in finite-width networks capable of feature learning—an explicit, data-dependent expression for generalization error dominated by the spectral structure of the target function. This expression reveals power-law relationships governing how generalization error scales with network width, sample size, and regularization strength, identifies multiple scaling regimes and their phase-transition boundaries (such as the interpolation threshold), and demonstrates that the spectral structure of the data fundamentally determines the exponent of the generalization power law.