🤖 AI Summary
To address the high overhead of dynamic data repartitioning in multi-core HPC systems under time-varying workloads, this paper proposes a lightweight, hierarchical partitioning method jointly driven by geometric and statistical principles. The method integrates space-filling curve ordering, greedy knapsack-based load balancing, and hierarchical data decomposition to support efficient dynamic partitioning of 2D/3D structured grids, point sets, and general graphs. It introduces, for the first time, an adaptive repartitioning mechanism guided by real-time feedback on data distribution, substantially reducing computational and communication overhead in frequently updated scenarios. Implemented via a hybrid parallel programming model (MPI + OpenMP) on modern many-core architectures, experimental results demonstrate a 3.2–5.7× speedup in partitioning time and a load imbalance ratio below 3.1%. This approach provides timely, low-overhead data partitioning support for parallel algorithms in large-scale scientific computing.
📝 Abstract
This article describes a geometric partitioning software that can be used for quick computation of data partitions on many-core HPC machines. It is most suited for dynamic applications with load distributions that vary with time. Partitioning costs were minimized with a lot of care, to tolerate frequent adjustments to the load distribution. The partitioning algorithm uses both geometry as well as statistics collected from the data distribution. The implementation is based on a hybrid programming model that is both distributed and multi-threaded. Partitions are computed by a hierarchical data decomposition, followed by data ordering using space-filling curves and greedy knapsack. This software was primarily used for partitioning 2 and 3 dimensional meshes in scientific computing. It was also used to solve point-location problems and for partitioning general graphs. The experiments described in this paper provide useful performance data for important parallel algorithms on a HPC machine built using a recent many-core processor designed for data-intensive applications by providing large on-chip memory.