Tensor-on-tensor Regression Neural Networks for Process Modeling with High-dimensional Data

2025-10-06

AI Summary

Modeling high-dimensional heterogeneous sensor data (e.g., images, point clouds) faces dual challenges: loss of intrinsic tensor structure during flattening and insufficient nonlinear expressivity. Method: This paper proposes a Structure-Preserving Tensor Neural Network (STNN), the first deep learning framework that embeds tensor-to-tensor regression end-to-end while preserving the native multiway structure of both inputs and outputs. STNN integrates multilinear decomposition, learnable tensor-kernel transformations, tensor contractions, and element-wise nonlinear activations to enable geometry-aware, nonlinear modeling. Contribution/Results: Evaluated on multiple industrial process modeling tasks, STNN consistently outperforms conventional linear tensor regression and flattened neural networks, achieving significant improvements in prediction accuracy, parameter efficiency, and generalization capability.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Modern sensing and metrology systems now stream terabytes of heterogeneous, high-dimensional (HD) data profiles, images, and dense point clouds, whose natural representation is multi-way tensors. Understanding such data requires regression models that preserve tensor geometry, yet remain expressive enough to capture the pronounced nonlinear interactions that dominate many industrial and mechanical processes. Existing tensor-based regressors meet the first requirement but remain essentially linear. Conversely, conventional neural networks offer nonlinearity only after flattening, thereby discarding spatial structure and incurring prohibitive parameter counts. This paper introduces a Tensor-on-Tensor Regression Neural Network (TRNN) that unifies these two paradigms.

Problem

• Modeling high-dimensional tensor data while preserving geometric structure
• Capturing nonlinear interactions in industrial processes with tensor inputs
• Overcoming limitations of linear tensor regressors and flattened neural networks

Innovation

• Tensor-on-tensor regression preserving tensor geometry
• Neural network capturing nonlinear interactions in processes
• Unifying tensor-based regressors with neural networks