Towards Portability at Scale: A Cross-Architecture Performance Evaluation of a GPU-enabled Shallow Water Solver

2025-11-02

AI Summary

To address the urgent need for high-resolution, real-time hydrological simulation under climate change, this work designs and implements SERGHEI-SWE—a cross-architecture GPU-accelerated shallow water equation (SWE) solver. Leveraging the Kokkos unified programming model, it integrates team-level parallel abstractions with architecture-specific optimizations to ensure high-performance portability across NVIDIA, AMD, and Intel GPUs. The implementation is guided by Roofline model analysis, employs mixed-precision memory management, and undergoes rigorous strong and weak scaling evaluation. At a scale of 2,048 GPUs, SERGHEI-SWE achieves >90% parallel efficiency and a 32× speedup over baseline, with significantly improved memory bandwidth utilization. This work represents the first large-scale, high-fidelity, highly scalable SWE simulation demonstrated across multi-vendor GPU architectures. It establishes a reusable, performance-portable paradigm for Earth system modeling on heterogeneous HPC platforms.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Current climate change has posed a grand challenge in the field of numerical modeling due to its complex, multiscale dynamics. In hydrological modeling, the increasing demand for high-resolution, real-time simulations has led to the adoption of GPU-accelerated platforms and performance portable programming frameworks such as Kokkos. In this work, we present a comprehensive performance study of the SERGHEI-SWE solver, a shallow water equations code, across four state-of-the-art heterogeneous HPC systems: Frontier (AMD MI250X), JUWELS Booster (NVIDIA A100), JEDI (NVIDIA H100), and Aurora (Intel Max 1550). We assess strong scaling up to 1024 GPUs and weak scaling upwards of 2048 GPUs, demonstrating consistent scalability with a speedup of 32 and an efficiency upwards of 90% for most almost all the test range. Roofline analysis reveals that memory bandwidth is the dominant performance bottleneck, with key solver kernels residing in the memory-bound region. To evaluate performance portability, we apply both harmonic and arithmetic mean-based metrics while varying problem size. Results indicate that while SERGHEI-SWE achieves portability across devices with tuned problem sizes (<70%), there is room for kernel optimization within the solver with more granular control of the architecture specifically by using Kokkos teams and architecture specific tunable parameters. These findings position SERGHEI-SWE as a robust, scalable, and portable simulation tool for large-scale geophysical applications under evolving HPC architectures with potential to enhance its performance.

Problem

• Evaluating cross-architecture performance of GPU-accelerated shallow water solver
• Assessing scalability and efficiency across four heterogeneous HPC systems
• Analyzing performance portability and identifying optimization opportunities for kernels

Innovation

• GPU-accelerated shallow water solver using Kokkos
• Performance portability across four HPC architectures
• Scalability demonstrated up to 2048 GPUs