Finite-Precision Gram-Schmidt Walks

2026-10-06

AI Summary

This study addresses the failure of theoretical guarantees and trajectory deviations in the Gram-Schmidt random walk algorithm under finite precision due to accumulated numerical errors. Leveraging moment generating function (MGF) analysis and numerical stability theory, we investigate the concentration of bounded, history-dependent perturbation errors and design multi-scenario ablation experiments to evaluate performance across varying precisions and scales. This work establishes the first rigorous theoretical guarantees for this algorithm under finite precision. We derive a corrected MGF bound that quantifies the impact of bias errors on frozen coordinates and subsequent trajectories, proving the inevitability of error accumulation. Furthermore, we construct full column rank instances revealing a deviation lower bound of order $\min\{n^2\epsilon, n\}$. Notably, the original exact bounds are recovered as the error approaches zero.

Abstract

The Gram-Schmidt Walk is a randomized vector-balancing algorithm whose subgaussian guarantees support applications in discrepancy, experimental design, and data compression; however, these theoretical guarantees are established in exact arithmetic, whereas implementations must approximate least-squares directions, boundary updates, and sampling probabilities in finite precision. This is important because small numerical errors can change which coordinates freeze and thereby alter the subsequent trajectory. We analyze the concentration of the perturbed Gram-Schmidt walk directly under bounded, potentially biased and history-dependent errors. For $n$ input vectors of Euclidean norm at most one, we obtain a modified MGF bound depending on key error sources which recovers the original result as the error goes to zero. We also construct a full-column-rank instance in which bounded update errors produce bias of order $\min\{n^2\varepsilon,n\}$, showing that updates accumulate error unavoidably under this model. Finally, we validate our findings in a variety of settings by ablating on the bit precision and problem size.

Problem

• Gram-Schmidt Walk
• finite precision
• vector balancing
• numerical errors
• subgaussian guarantees

Innovation

• Gram-Schmidt Walk
• finite-precision arithmetic
• randomized vector balancing
• moment generating function bound
• error accumulation