FCN: Fusing Exponential and Linear Cross Network for Click-Through Rate Prediction

2024-07-18

AI Summary

To address four key limitations of Deep & Cross Network (DCN)-style models in click-through rate (CTR) prediction—weak explicit high-order feature interaction, strong noise sensitivity, insufficient supervision, and poor interpretability—this paper proposes the Dual-Path Cross Network (DP-CrossNet). DP-CrossNet introduces a novel dual-path architecture comprising a Linear Cross Network (LCN) and an Exponential Cross Network (ECN), enabling explicit, interpretable multi-order feature interactions. A self-mask mechanism is incorporated to progressively suppress noise and reduce parameter count by 50%. Furthermore, a Tri-Binary Cross-Entropy (Tri-BCE) multi-objective loss function provides differentiated supervision for the two paths. Crucially, DP-CrossNet completely eliminates the DNN backbone, relying solely on explicit cross-layer structures. Extensive experiments on six benchmark datasets demonstrate that DP-CrossNet consistently outperforms state-of-the-art methods, achieving significant gains in prediction accuracy, training efficiency, and model interpretability.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

As an important modeling paradigm in click-through rate (CTR) prediction, the Deep&Cross Network (DCN) and its derivative models have gained widespread recognition primarily due to their success in a trade-off between computational cost and performance. This paradigm employs a cross network to explicitly model feature interactions with linear growth, while leveraging deep neural networks (DNN) to implicitly capture higher-order feature interactions. However, these models still face several key limitations: (1) The performance of existing explicit feature interaction methods lags behind that of implicit DNN, resulting in overall model performance being dominated by the DNN; (2) While these models claim to capture high-order feature interactions, they often overlook potential noise within these interactions; (3) The learning process for different interaction network branches lacks appropriate supervision signals; and (4) The high-order feature interactions captured by these models are often implicit and non-interpretable due to their reliance on DNN. To address the identified limitations, this paper proposes a novel model, called Fusing Cross Network (FCN), along with two sub-networks: Linear Cross Network (LCN) and Exponential Cross Network (ECN). FCN explicitly captures feature interactions with both linear and exponential growth, eliminating the need to rely on implicit DNN. Moreover, we introduce the Self-Mask operation to filter noise layer by layer and reduce the number of parameters in the cross network by half. To effectively train these two cross networks, we propose a simple yet effective loss function called Tri-BCE, which provides tailored supervision signals for each network. We evaluate the effectiveness, efficiency, and interpretability of FCN on six benchmark datasets. Furthermore, by integrating LCN and ECN, FCN achieves a new state-of-the-art performance.

Problem

• Improving explicit feature interaction in CTR prediction models
• Reducing noise in high-order feature interactions
• Enhancing interpretability of feature interactions

Innovation

• Fuses linear and exponential cross networks
• Introduces Self-Mask for noise filtering
• Uses Tri-BCE loss for tailored supervision