Nearest Neighbors GParareal: Improving Scalability of Gaussian Processes for Parallel-in-Time Solvers

2024-05-20

arXiv.org

AI Summary

To address the poor scalability of Gaussian process (GP)-driven parallel-in-time (PinT) methods—such as GParareal—in high-dimensional systems and large-scale parallel environments, this paper proposes nnGParareal. The key innovation is the first integration of nearest-neighbor sampling into the GP-PinT framework, combined with adaptive sample selection and neighbor-based data compression. This reduces the GP model complexity from O(N³) to O(N log N) while preserving numerical accuracy. Theoretical analysis establishes rigorous error bounds and guarantees on parallel speedup. Extensive experiments across nine representative dynamical systems—including stiff, chaotic, and high-dimensional cases—demonstrate that nnGParareal significantly outperforms both GParareal and classical Parareal: it achieves faster convergence, higher parallel efficiency, and superior strong scaling behavior.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Hardware-aware ML

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

With the advent of supercomputers, multi-processor environments and parallel-in-time (PinT) algorithms offer ways to solve initial value problems for ordinary and partial differential equations (ODEs and PDEs) over long time intervals, a task often unfeasible with sequential solvers within realistic time frames. A recent approach, GParareal, combines Gaussian Processes with traditional PinT methodology (Parareal) to achieve faster parallel speed-ups. The method is known to outperform Parareal for low-dimensional ODEs and a limited number of computer cores. Here, we present Nearest Neighbors GParareal (nnGParareal), a novel data-enriched PinT integration algorithm. nnGParareal builds upon GParareal by improving its scalability properties for higher-dimensional systems and increased processor count. Through data reduction, the model complexity is reduced from cubic to log-linear in the sample size, yielding a fast and automated procedure to integrate initial value problems over long time intervals. First, we provide both an upper bound for the error and theoretical details on the speed-up benefits. Then, we empirically illustrate the superior performance of nnGParareal, compared to GParareal and Parareal, on nine different systems with unique features (e.g., stiff, chaotic, high-dimensional, or challenging-to-learn systems).

Problem

• Improving scalability of Gaussian Processes in PinT solvers
• Reducing model complexity for high-dimensional systems
• Enhancing speed and automation in long-time integration

Innovation

• Combines Gaussian Processes with PinT methodology
• Reduces model complexity via data reduction
• Improves scalability for high-dimensional systems