Linear-Scaling Tensor Train Sketching

2026-03-11

AI Summary

This work proposes a block-sparse tensor train (BSTT) sketching method to overcome the exponential computational complexity in tensor order \(d\) inherent in existing tensor train (TT) random projection approaches. By introducing tunable parameters \(P\) and \(R\), the BSTT framework interpolates between Khatri–Rao and Gaussian TT sketches, achieving—for the first time—oblivious subspace embedding (OSE) and oblivious subspace inclusion (OSI) guarantees that scale linearly in both \(d\) and the subspace dimension \(r\). The method integrates structured random projections, TT decomposition, and efficient QB and randomized rounding algorithms. Experiments on synthetic tensors, Hadamard products, and quantum chemistry tasks demonstrate near-optimal error bounds and substantial acceleration, effectively breaking the exponential bottleneck of conventional techniques.

Technology Category

Machine Learning: Matrix & Tensor MethodsSearch and Optimization: Non-convex OptimizationReasoning under Uncertainty: Stochastic Optimization

Application Category

Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphsSemantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologiesSearch and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We introduce the Block Sparse Tensor Train (BSTT) sketch, a structured random projection tailored to the tensor train (TT) format that unifies existing TT-adapted sketching operators. By varying two integer parameters $P$ and $R$, BSTT interpolates between the Khatri-Rao sketch ($R=1$) and the Gaussian TT sketch ($P=1$). We prove that BSTT satisfies an oblivious subspace embedding (OSE) property with parameters $R = \mathcal{O}(d(r+\log 1/δ))$ and $P = \mathcal{O}(\varepsilon^{-2})$, and an oblivious subspace injection (OSI) property under the condition $R = \mathcal{O}(d)$ and $P = \mathcal{O}(\varepsilon^{-2}(r + \log r/δ))$. Both guarantees depend only linearly on the tensor order $d$ and on the subspace dimension $r$, in contrast to prior constructions that suffer from exponential scaling in $d$. As direct consequences, we derive quasi-optimal error bounds for the QB factorization and randomized TT rounding. The theoretical results are supported by numerical experiments on synthetic tensors, Hadamard products, and a quantum chemistry application.

Problem

• tensor train
• random projection
• subspace embedding
• linear scaling
• sketching

Innovation

• Tensor Train
• Sketching
• Oblivious Subspace Embedding
• Linear Scaling
• Randomized Numerical Linear Algebra