tensorFM: Low-Rank Approximations of Cross-Order Feature Interactions

2026-02-16

AI Summary

This work addresses the challenge of simultaneously achieving high modeling efficiency and strong representational capacity when capturing high-order feature interactions in tabular classification data. To this end, we propose a unified modeling framework based on low-rank tensor decomposition, which generalizes the field-weighted factorization machine. By leveraging low-rank approximations, our method efficiently captures cross-order feature interactions while maintaining low computational complexity and inference latency, thereby significantly enhancing model expressiveness. Experimental results demonstrate that the proposed approach achieves performance comparable to state-of-the-art models on multiple click-through rate prediction benchmarks, with notably faster inference speed, making it well-suited for real-time applications such as online advertising.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We address prediction problems on tabular categorical data, where each instance is defined by multiple categorical attributes, each taking values from a finite set. These attributes are often referred to as fields, and their categorical values as features. Such problems frequently arise in practical applications, including click-through rate prediction and social sciences. We introduce and analyze {tensorFM}, a new model that efficiently captures high-order interactions between attributes via a low-rank tensor approximation representing the strength of these interactions. Our model generalizes field-weighted factorization machines. Empirically, tensorFM demonstrates competitive performance with state-of-the-art methods. Additionally, its low latency makes it well-suited for time-sensitive applications, such as online advertising.

Problem

• tabular categorical data
• feature interactions
• high-order interactions
• click-through rate prediction
• factorization machines

Innovation

• tensorFM
• low-rank tensor approximation
• high-order feature interactions
• factorization machines
• tabular categorical data