An inherently parallel ℋ2-ULV factorization for solving dense linear systems on GPUs

2024-04-03

The international journal of high performance computing applications

AI Summary

ℋ² matrices suffer from GPU parallelization bottlenecks due to structural heterogeneity, strong inter-block dependencies, and load imbalance. Method: This paper proposes an ℋ²-ULV decomposition method that eliminates trailing submatrix dependencies—enabling intrinsic parallelism in ℋ² matrix factorization for the first time. By reformulating the ULV framework and integrating CUDA asynchronous streams with fine-grained task scheduling, inter-block data dependencies are removed and dynamic load balancing is supported. Contribution/Results: The algorithm achieves O(N) computational complexity and delivers superlinear speedup across multiple GPUs. Compared to state-of-the-art ℋ² solvers, it accelerates performance by up to 5.2×, significantly reducing runtime overhead induced by low-rank subblocks. This work provides an efficient, scalable GPU-accelerated solution for dense linear systems.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Hierarchical low-rank approximation of dense matrices can reduce the complexity of their factorization from [Formula: see text] to [Formula: see text]. However, the complex structure of such hierarchical matrices makes them difficult to parallelize. The block size and ranks can vary between the sub-blocks, which creates load imbalance. The dependency between the sub-blocks during factorization results in serialization. Since many sub-blocks are low-rank, their small computational load exposes the overhead of runtime systems. The combination of these factors makes it challenging to implement these methods on GPUs. In this work, we show that dense matrices can be factorized with linear complexity, while extracting the potential parallelism of GPUs. This is made possible through the [Formula: see text]-ULV factorization, which removes the dependency on trailing sub-matrices.

Problem

• Parallel H2-ULV factorization on GPUs
• Hierarchical low-rank matrix complexity reduction
• Load imbalance and serialization challenges

Innovation

• H2-ULV factorization
• parallel GPU processing
• linear complexity reduction