orthogonalization methods

本文通过构建具有最小支撑的张量基，解决了成对比较理论中加性一致子空间的正交基问题，并提出了新的对数、Saaty和SVD投影公式。

To address numerical instability in block orthogonalization within s-step GMRES, this work introduces— for the first time—the integration of randomized sketching into the block QR orthogonalization process, eliminating the need for classical reorthogonalization while rigorously bounding the overall orthogonality error of basis vectors at the machine-precision level. The method synergistically combines randomized projection with distributed block QR and is implemented within the Trilinos framework to enable GPU acceleration (A100) and large-scale parallel scalability. Experiments on the Perlmutter supercomputer demonstrate substantial improvements in both orthogonalization and overall solver numerical stability, with negligible overhead in execution time. This work establishes a novel orthogonalization paradigm for highly parallel Krylov methods that simultaneously ensures high numerical accuracy and computational efficiency.

This study addresses accuracy and consistency issues in the numerical construction of the Karhunen–Loève expansion (KLE) arising from discretization, quadrature rules, and finite sample sizes. It establishes an algebraic equivalence between the spectral decomposition of the Fredholm integral equation and the singular value decomposition (SVD) of a weighted sample covariance matrix, thereby unifying model-driven and data-driven KLE frameworks. The work innovatively constructs the covariance function on a non-simply-connected three-dimensional toroidal domain using the shortest interior path distance, and implements the approach numerically with unstructured meshes and Gaussian quadrature. Experiments demonstrate that, in a one-dimensional benchmark problem, SVD-based eigenvalue estimates and empirical KL coefficients converge to the theoretical 𝒩(0,1) distribution. In two-dimensional irregular and three-dimensional toroidal domains, the study systematically quantifies the combined influence of discretization strategy, quadrature accuracy, and sample size on KLE reconstruction error.

This work addresses the issue of uncontrolled reconstruction errors in SVD-based compression of large collections of matrices when heuristic grouping is employed prior to concatenation. To overcome this limitation, the authors propose a theory-driven compressive clustering framework grounded in spectral analysis of horizontally concatenated matrices. They establish, for the first time, a globally provable upper bound on SVD reconstruction error and derive two novel spectral bounds based on a lower bound for singular value growth. Building upon these theoretical guarantees, they design three clustering algorithms with explicit error control, integrated with incremental approximate SVD to efficiently estimate compression error without explicitly forming the full concatenated matrix. The resulting approach achieves a favorable balance among speed, accuracy, and scalability, significantly enhancing the reliability and practicality of SVD compression in applications such as multi-view learning, signal processing, and neural network compression.

This work investigates the impact of matrix concatenation operations on singular value spectra, aiming to bridge a theoretical gap in the structural stability of SVDs under concatenation. Addressing the central question—“How are the singular values of a concatenated matrix determined by those of its constituent submatrices?”—we extend Weyl’s inequality to block-wise concatenation for the first time, establishing a quantitative analytical framework grounded in matrix perturbation theory and norm inequalities. We derive computable upper bounds on singular value deviations and rigorously prove that dominant singular values remain stable when the operator norms of the submatrices are comparable. This result provides theoretical guarantees and principled design guidance for low-rank approximation, robust matrix clustering, and compression algorithms.

本文介绍了一种通过计算交换算子的联合特征向量来解决多项式根和张量分解问题的方法，并分析了其多重结构。

CORAM通过在奇异值分解框架下合并模型权重的正交旋转来解决模型合并问题，使用放大系数和分散切片技术以提高性能。

This work addresses the challenges of discovering differential equations, approximating functions, and estimating high-dimensional integrals from noisy, non-uniformly sampled data by introducing Sparse Orthogonal Regression Technique (SORT). The method reformulates equation discovery as a spectral coefficient learning problem, directly inferring coefficients of an orthogonal basis expansion from observational data via L1 regularization—without requiring a predefined symbolic library, explicit numerical integration, or inner product evaluations. Its core innovation lies in treating basis function design as the central modeling choice, enabling consistent model order scaling and multi-task reusability. Experiments demonstrate that when the basis functions align with the underlying problem structure, SORT matches or outperforms existing approaches under sparse sampling, noisy derivatives, and representation mismatch, while low-order dominant coefficients remain stable as model complexity increases.

This study addresses the precision-efficiency imbalance in existing matrix optimizers, where Newton-Schulz orthogonalization relies on fixed polynomials that disregard variations in singular value spectra. We propose an adaptive orthogonalization method that leverages cost-free scalar reductions of the intra-iteration Gram matrix to estimate spectral distributions. This enables dynamic selection of optimal polynomials without additional matrix multiplications, shifting from conservative worst-case designs to real-time data-driven adaptive computation. During GPT pretraining, our approach significantly reduces validation loss, requiring fewer iterations at equivalent precision or yielding superior orthogonalization quality under identical computational budgets.

Traditional proofs of Singular Value Decomposition (SVD) rely on the spectral theorem, lacking geometric intuition and failing to reveal deep connections with machine learning algorithms. This work reconstructs SVD from an ellipsoidal geometry perspective, transforming the recursive process of maximizing stretch into an algorithmic mechanism. By integrating geometric constructions, gradient descent stopping criteria, and duality analysis of kernel methods, it achieves a "proof-as-algorithm" paradigm that derives SVD without presupposing the spectral theorem. Furthermore, this study establishes explicit mappings between SVD and core algorithms such as Principal Component Analysis (PCA) and PageRank, thereby unifying the foundational logic of linear algebra and machine learning. Finally, it provides a pedagogical framework amenable to manual verification alongside a discussion of theoretical boundaries.