Error Analysis of Sum-Product Algorithms under Stochastic Rounding

2024-11-19

arXiv.org

AI Summary

This paper addresses the forward error analysis of sum-product algorithms under stochastic rounding (SR). We propose a probabilistic error bounding method grounded in martingale theory. Our key contributions are threefold: (1) We introduce the first automated martingale construction framework tailored to multilinear computational structures—encompassing addition, subtraction, multiplication, and intermediate result reuse; (2) We extend SR error analysis to algorithms with structural reuse, notably Karatsuba polynomial multiplication—previously unaddressed in SR literature; (3) Leveraging the Azuma–Hoeffding inequality, we derive a tight probabilistic error bound of $O(sqrt{n},u)$, markedly improving upon the classical worst-case bound $O(n,u)$. Our framework uniformly recovers known error guarantees for pairwise summation and Horner’s method, and—crucially—provides the first rigorous SR error guarantee for Karatsuba multiplication.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

The quality of numerical computations can be measured through their forward error, for which finding good error bounds is challenging in general. For several algorithms and using stochastic rounding (SR), probabilistic analysis has been shown to be an effective alternative for obtaining tight error bounds. This analysis considers the distribution of errors and evaluates the algorithm's performance on average. Using martingales and the Azuma-Hoeffding inequality, it provides error bounds that are valid with a certain probability and in $mathcal{O}(sqrt{n}u)$ instead of deterministic worst-case bounds in $mathcal{O}(nu)$, where $n$ is the number of operations and $u$ is the unit roundoff.In this paper, we present a general method that automatically constructs a martingale for any computation scheme with multi-linear errors based on additions, subtractions, and multiplications. We apply this generalization to algorithms previously studied with SR, such as pairwise summation and the Horner algorithm, and prove equivalent results. We also analyze a previously unstudied algorithm, Karatsuba polynomial multiplication, which illustrates that the method can handle reused intermediate computations.

Problem

• Analyzing forward error bounds for numerical algorithms
• Developing probabilistic error analysis using stochastic rounding
• Generalizing martingale methods for multi-linear computations

Innovation

• Martingale-based error analysis for stochastic rounding
• Automatic martingale construction for multi-linear computations
• Generalized method handles reused intermediate computations