Improving Improved Kernel PLS

2026-07-17

AI Summary

This study addresses the computational inefficiency in calculating the X-rotation matrix \( R \) and Y-loading matrix \( Q \) within the Improved Kernel PLS (IKPLS) algorithm. To this end, it introduces two key optimizations: first, reformulating the computation of \( R \) into a directly evaluable form to enhance parallelism; second, revealing for the first time that \( Q \) can be exactly derived from intermediate quantities generated within the same iteration, thereby reducing its computational complexity from \( \Theta(KM) \) to \( \Theta(M) \) when \( M = 1 \) or \( 2 \leq M < K \), without any loss of numerical accuracy. Implementations leveraging NumPy (CPU) and JAX (GPU), combined with matrix algebra optimizations and hardware-aware parallelization strategies, achieve up to two orders of magnitude speedup in critical steps, yielding approximately 2× acceleration on CPU and up to 6× on GPU for full model fitting, and have been integrated into the open-source Python package `ikpls`.

Abstract

Improved Kernel Partial Least Squares (IKPLS) algorithms 1 and 2 are among the fastest PLS calibration algorithms. This article focuses on two shared steps, the computation of the $\mathbf{X}$ rotations, $\mathbf{R}$, and the $\mathbf{Y}$ loadings, $\mathbf{Q}$, and accelerates both. For $\mathbf{R}$, term-by-term accumulation is replaced by a direct evaluation strategy that requires the same number of multiplications but parallelizes better on modern hardware. For $\mathbf{Q}$, I identify - to the best of my knowledge, for the first time - equivalences showing that each $\mathbf{Y}$ loading is obtainable, up to explicitly derived constants, from quantities already computed earlier in the same iteration, and I exploit them in IKPLS to reduce the cost of each loading from $Θ\left(KM\right)$ to $Θ\left(M\right)$ operations whenever $M = 1$ or $2 \leq M < K$, with $K$ predictor variables (number of columns in $\mathbf{X}$) and $M$ response variables (number of columns in $\mathbf{Y}$). Both improvements provably yield exactly the same $\mathbf{W}$, $\mathbf{P}$, $\mathbf{Q}$, $\mathbf{R}$, and $\mathbf{T}$ as the original algorithms. Benchmarks with NumPy (CPU) and JAX (GPU) show speedups of up to two orders of magnitude for the isolated steps and of approximately $2\times$ (CPU) and $6\times$ (GPU) for entire fits. Both improvements are implemented in the free, open-source Python package \texttt{ikpls}.

Problem

• Kernel PLS
• IKPLS
• computational efficiency
• X rotations
• Y loadings

Innovation

• Improved Kernel PLS
• computational acceleration
• parallelization
• loadings computation
• algorithmic equivalence