efficient tensor operations

High-order tensor-weighted neural networks—such as the Fourier Neural Operator (FNO)—suffer from explosive memory consumption and inefficient training in scientific computing due to their high-dimensional parameter spaces. Method: This paper proposes an embedded gradient tensor decomposition optimization framework that, for the first time, integrates Tucker and CP decompositions directly into the optimization process. It performs low-rank gradient approximation and structure-preserving low-rank updates entirely within the tensor space, with theoretical convergence guarantees. The method requires no architectural modifications and is fully compatible with mainstream FNO variants. Results: Evaluated on PDE-solving tasks—including Navier–Stokes and Darcy Flow—the framework reduces GPU memory usage by up to 75% while preserving full accuracy. It significantly enhances scalability and training efficiency of high-fidelity scientific AI models without compromising solution quality.

Tensor operators consume over 90% of computational resources in LLMs and deep learning, yet manual optimization is time-consuming and exhibits poor portability across heterogeneous hardware (e.g., RISC-V, ARM, GPU). Method: This paper proposes the first hardware-primitive-aware, large-model-driven automatic operator generation framework. It innovatively injects hardware semantics into the LLM generation pipeline to jointly optimize operator structure and tunable parameters. The framework integrates hardware-aware prompt engineering, template-constrained decoding, a lightweight auto-tuner, and a multi-platform performance feedback loop, enabling zero-shot cross-architecture deployment. Contribution/Results: Experiments show a 1,291× speedup in operator generation over baseline LLMs; achieved 251% of OpenBLAS performance on RISC-V and 124% of cuBLAS performance on GPU; and reduced development effort by 200×.

This work investigates whether Tensor Cores deliver practical acceleration for memory-bound kernels—such as STREAM Scale, SpMV, and stencil computations—challenging recent studies that overestimate their performance in such scenarios. Method: The authors adopt a dual approach: (i) theoretically deriving the upper bound of double-precision speedup under GPU microarchitectural constraints (e.g., memory bandwidth and instruction scheduling overhead), and (ii) empirically validating across V100, A100, and H100 GPUs using both CUDA and WMMA APIs on representative memory-bound kernels. Contribution/Results: The analysis reveals a strict theoretical speedup ceiling of 1.33× for double-precision operations; all empirical measurements fall at or below this bound. The study refutes the efficacy of Tensor Cores in memory-bottlenecked workloads, attributing prior overestimations to neglect of bandwidth saturation and scheduling latency. It establishes the first principled theoretical foundation for Tensor Core applicability boundaries, providing a critical criterion for heterogeneous resource scheduling in GPU-accelerated computing.

In tensorized neural networks, initializing high-dimensional tensor layers often leads to parameter explosion or vanishing norms, while explicit parameter storage is infeasible due to memory constraints. Method: This paper proposes a norm-constrained finite initialization method that innovatively combines the Frobenius norm with partial linear element-wise norms, enabling iterative tensor normalization with intermediate computation reuse—applicable to fully connected, large-scale, and implicitly parameterized tensor layers. Contribution/Results: The method significantly mitigates gradient explosion and enhances training stability across diverse tensorized architectures. Its open-source Python implementation has been integrated into the i3BQuantum library—a quantum-inspired variational algorithm framework—demonstrating strong reusability, extensibility, and practical utility for scalable tensorized deep learning.

Optimizing tensor programs across GPU’s hierarchical compute architecture—kernels, thread blocks, and threads—remains challenging due to fragmented optimization scopes and insufficient cross-level coordination. To address this, we propose Mirage, the first multi-level super-optimizer explicitly designed for this hierarchy. Its core contributions are: (1) a unified intermediate representation, μGraph, enabling joint modeling of algebraic transformations, scheduling optimizations, and custom CUDA kernel generation; (2) abstraction-guided pruning and probabilistic equivalence verification, ensuring correctness while drastically improving search efficiency; and (3) end-to-end exploration of the multi-level scheduling space with automatic code generation. Evaluation on mainstream DNN models shows Mirage achieves 1.1–2.9× speedup over state-of-the-art optimizers including TVM and Ansor. The open-source implementation is publicly available.

This work addresses the high data collection costs and poor cross-hardware generalization of traditional deep learning compilers, which rely on large-scale offline datasets for tensor program optimization. To overcome these limitations, the authors propose TCL, a novel framework that integrates a representativeness-diversity-uncertainty joint active sampling strategy, a lightweight Mamba-based cost model, and a cross-platform continual knowledge distillation mechanism. This design substantially reduces dependency on training data while enabling rapid optimization across heterogeneous hardware such as CPUs and GPUs. Experimental results demonstrate that TCL achieves 16.8× and 12.48× faster average tuning speeds than Tenset-MLP on CPU and GPU platforms, respectively, while attaining inference latencies of only 1.20× and 1.13× those of Tenset-MLP.

This work addresses the “memory wall” bottleneck that hinders large language model deployment on edge devices, where data movement latency far exceeds computational capacity. The authors propose a software-based “virtual tensor core” tailored for ARM64 architectures, featuring a Tensor Virtualization Layout (TVL) that achieves 100% cache line utilization. By integrating direct memory mapping (mmap), hand-optimized NEON SIMD kernels, and software-defined DMA, the system enables a zero-copy, initialization-free inference engine. This approach substantially improves cache efficiency, delivering stable throughput exceeding 60 tokens per second on an Apple Silicon M2 for a 110-million-parameter model—meeting the 200 ms psycholinguistic latency threshold. The project provides an open-source, portable, and deterministic reference implementation for edge AI inference.

This work addresses the challenge of deploying mainstream deep learning frameworks in resource-constrained environments, where their large size and lack of lightweight yet fully featured alternatives pose significant limitations. To this end, we propose and implement a lightweight tensor computation library built in Rust, leveraging its performance and memory safety guarantees to construct an efficient computational engine. The system exposes a PyTorch-like Python interface via PyO3, supporting essential features including n-dimensional tensors, dynamic computation graphs, reverse-mode automatic differentiation, neural network layers, and optimizers. The resulting installable package occupies only a few megabytes—orders of magnitude smaller than PyTorch or TensorFlow—while retaining the core capabilities necessary for research and development on CPU-based systems.

This work addresses the limited flexibility in distributed programming for large language model scaling and the inefficiency of existing tensor compilers in handling the complex memory hierarchies of heterogeneous clusters. To overcome these challenges, the authors propose a scalable block-level compiler featuring a novel three-tier hierarchical abstraction—Core, Device, and Task—that uniformly supports diverse parallelization strategies, automatically optimizes intra- and inter-node communication, and enables efficient code generation across both NVIDIA and AMD platforms. When integrated into vLLM, the compiler achieves 5%–30% end-to-end inference speedup and over 10% improvement in training model FLOPs utilization (MFU), translating to approximately 500,000 GPU hours saved per month. The system has been deployed in enterprise settings, delivering over 20% inference performance gains.