ML-Based Optimum Number of CUDA Streams for the GPU Implementation of the Tridiagonal Partition Method

2025-01-10

AI Summary

Optimizing the number of CUDA streams for block tridiagonal solvers on GPUs remains challenging due to complex hardware-dependent trade-offs between concurrency, memory bandwidth saturation, and kernel launch overhead. Method: This paper introduces a hardware-aware, fine-grained time-complexity model and the first machine learning–inspired approach for automatic stream count prediction. It proposes a nonlinear stream overhead model and a stream-interleaving overlap cost model, integrated with regression analysis—including nonlinear fitting—and statistical validation to derive a generalizable, interpretable optimal stream count predictor. Results: Experimental evaluation on sparse linear algebra equation (SLAE) systems of varying scales demonstrates bounded prediction error, up to 18.7% improvement in GPU computational throughput, and significantly enhanced resource utilization. The method establishes a novel paradigm for adaptive, self-tuning heterogeneous parallel solvers.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Solvers and Tools
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents a heuristic for finding the optimum number of CUDA streams by using tools common to the modern AI-oriented approaches and applied to the parallel partition algorithm. A time complexity model for the GPU realization of the partition method is built. Further, a refined time complexity model for the partition algorithm being executed on multiple CUDA streams is formulated. Computational experiments for different SLAE sizes are conducted, and the optimum number of CUDA streams for each of them is found empirically. Based on the collected data a model for the sum of the times for the non-dominant GPU operations (that take part in the stream overlap) is formulated using regression analysis. A fitting non-linear model for the overhead time connected with the creation of CUDA streams is created. Statistical analysis is done for all the built models. An algorithm for finding the optimum number of CUDA streams is formulated. Using this algorithm, together with the two models mentioned above, predictions for the optimum number of CUDA streams are made. Comparing the predicted values with the actual data, the algorithm is deemed to be acceptably good.

Problem

• CUDA Streams Optimization
• Tridiagonal Solver
• GPU Efficiency

Innovation

• Machine Learning
• CUDA Stream Optimization
• Prediction Accuracy