performance profiling

Modern software systems frequently exhibit application-level resource contention bottlenecks—such as blocking on custom application events—that evade detection by conventional performance profilers due to complex dependencies and bespoke resource management. To address this, we propose OmniResource Profiling, the first method to jointly leverage system-level metrics and application-level event-waiting relationships. It employs a lightweight LLM-assisted static analysis to automatically identify custom resources and cross-execution-trace runtime variable comparison for precise root-cause localization. Evaluated on 12 known performance issues across five real-world applications, OmniResource achieves 100% diagnostic accuracy and uncovers two previously undetected bottlenecks. Crucially, it requires no intrusive instrumentation, balancing high precision with practical deployability. This work delivers the first end-to-end solution for application-level resource contention analysis.

Accurately identifying computational, memory bandwidth, and memory latency bottlenecks—and quantifying associated resource slack—is critical yet challenging for HPC application performance tuning. This paper introduces the first model-agnostic, instruction-level precise noise-injection framework for bottleneck analysis. Leveraging the LLVM toolchain, it selectively injects computational or memory-access noise instructions to decouple the impact of each resource constraint, enabling fine-grained bottleneck classification and quantitative slack measurement. Unlike prior approaches, it requires no hardware modeling assumptions and is portable across diverse architectures. Evaluated on heterogeneous memory systems—including HBM and DDR—it demonstrates robust effectiveness. The method significantly improves the precision of optimization decisions and hardware selection guidance, addressing key limitations of existing tools in slack quantification and root-cause attribution of performance bottlenecks.

A lack of systematic methodologies for cross-platform performance and scalability comparison across heterogeneous HPC systems hinders fair and reproducible evaluation. Method: This paper proposes a unified cross-platform evaluation paradigm centered on the single compute node as the baseline unit. It integrates node-level performance measurement, weak/strong scaling analysis, and normalized metric comparison into a standardized experimental design, execution, and reporting workflow. A general-purpose validation framework is developed and empirically applied across diverse architectures—including CPUs, GPUs, and heterogeneous accelerators. Contributions/Results: (1) Establishes the single node as the minimal comparable unit for cross-platform assessment; (2) Provides a reusable, standardized evaluation template and integrated toolchain; (3) Significantly improves consistency, reproducibility, and interpretability of performance evaluation across heterogeneous HPC platforms.

Low-level systems (e.g., network stacks) resist efficient compile-time specialization for dynamic workloads and runtime environments due to high implementation complexity, difficulty in predicting optimal strategies, and continuously evolving conditions. This paper introduces Iridescent—the first online, automatic, measurement-driven JIT code specialization framework for low-level systems. Developers annotate only lightweight specialization points; the system autonomously explores and deploys optimal specialization strategies via JIT compilation and dynamic binary rewriting, guided by real-time performance feedback (e.g., latency, throughput), eliminating reliance on static compilation and manual modeling. Evaluated on real-world network stacks, Iridescent achieves significant performance improvements—up to 2.3× latency reduction and 1.8× throughput gain—while imposing minimal developer overhead. Crucially, it enables continuous optimization in response to dynamic workload shifts and heterogeneous hardware platforms, demonstrating robust adaptability across diverse deployment scenarios.

Existing benchmarks inadequately capture the complexity of performance optimization in real-world codebases, often neglecting trade-offs between runtime and memory usage, measurement noise, and input variability. To address this gap, this work proposes SWE-Pro—the first repository-scale benchmark for performance optimization—constructed from expert-driven optimization cases across 102 open-source projects. SWE-Pro introduces multidimensional evaluation metrics, including parameterized testing, noise-aware measurement protocols, and Time-Weighted Memory Usage (TWMU), to holistically reflect practical engineering challenges. Experimental results demonstrate that current large language models exhibit limited effectiveness on this benchmark, achieving negligible runtime improvements and virtually no memory optimization, whereas expert solutions yield an average speedup of 15.5× and a 171.3× reduction in peak memory consumption, underscoring both the benchmark’s realism and its difficulty.

This work addresses the challenge of identifying subtle performance bugs in database management systems, which are often difficult to detect and lack systematic validation methods for optimization logic. The authors propose Branch Flipping Analysis (BFA), a novel white-box, fine-grained detection paradigm that combines static analysis with dynamic execution to precisely control and compare performance differences between code paths with optimizations enabled versus disabled. By directly manipulating conditional branches related to query optimizations, BFA overcomes the limitations of traditional black-box testing. The approach has been successfully applied to four major database systems—including PostgreSQL and MySQL—uncovering 21 previously unknown performance defects, thereby demonstrating its effectiveness and broad applicability.

Existing static benchmarks inadequately assess the real-world capabilities of large language model (LLM) agents under the complex conditions of production environments—particularly regarding long-horizon execution, multi-tool coordination, and dependency management. This work proposes RAMP, the first runtime evaluation framework tailored for production settings, built upon the YatCC platform. RAMP dynamically evaluates agent performance on realistic software compilation workloads through a unified runtime architecture, standardized execution interfaces, and sequentially dependent task designs, while incorporating a staged recovery mechanism to analyze behavior under partial failures. Coupled with multidimensional utility metrics, RAMP jointly evaluates both process efficiency and output quality. Experiments across 15 prominent models reveal a sharp decline in task completion rates—from 100% initially to only 20% in the final stage—with no model successfully completing the entire workflow, exposing systemic error propagation and resource consumption disparities up to three orders of magnitude, thereby demonstrating that conventional benchmarks substantially overestimate practical agent capabilities.

This work addresses the challenge of memory inefficiencies—such as redundant allocations and suboptimal usage—in large-scale software systems, which often lead to significant resource waste and performance degradation. Existing optimization approaches lack end-to-end automation and struggle to scale to codebases exceeding hundreds of millions of lines. To overcome this, we propose MOA, a novel framework that integrates multi-agent large language models with performance profiling data. MOA employs three coordinated agents—Analyzer, Checker Generator, and Patcher—to automatically detect memory anti-patterns, synthesize static checkers, and generate state-machine-guided, semantics-preserving patches. Evaluated on OpenHarmony’s C/C++ codebase (>100 million lines), MOA identified 13 memory anti-patterns (9 previously unknown), pinpointed over 10,000 inefficiency instances, and produced 769 patches with a 92.5% expert acceptance rate, reducing heap memory usage by 42.2% and binary size by 10.6% on average.

This work addresses the pervasive yet elusive performance issues in mobile applications—such as UI jank and thermal throttling—caused by native libraries compiled with low-level optimization flags (e.g., O0/O1), which are notoriously difficult to detect. To tackle this, we propose OptDetect, the first end-to-end framework capable of identifying mixed optimization levels without requiring source code or build metadata. OptDetect leverages binary disassembly, machine learning–driven block-level optimization classification, and a weighted scoring aggregation scheme to accurately pinpoint under-optimized code segments. Evaluation on 830 popular apps reveals that 91.7% are affected by poorly optimized third-party libraries. Post-remediation results demonstrate up to a 63% reduction in CPU instructions, a median 42% drop in user-reported performance complaints, and a 0.14-point increase in app store ratings, confirming both the efficacy and industry-wide applicability of our approach.

This work addresses the limitation of existing energy profiling tools, which only support coarse-grained runtime evaluation and cannot provide fine-grained energy estimates for code constructs—such as functions or loops—during the design phase. To bridge this gap, we propose EnCoDe, the first method enabling block-level energy estimation at design time. EnCoDe leverages PowerLens, a custom-built high-precision measurement tool that captures sub-millisecond energy traces, and combines them with static code features—including structural properties, complexity, density, and contextual information—to train machine learning models that predict energy consumption and identify energy hotspots. Using a novel dataset of over 18,000 Python programs annotated at the code-block level, our regression model achieves an R² of 0.75, and hotspot classification accuracy reaches 80.6%, enabling stable, execution-free, fine-grained energy analysis.