On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective

2024-11-20

arXiv.org

AI Summary

To address storage bottlenecks and costly retraining induced by large-scale embedding tables in content-based recommendation systems deployed on resource-constrained edge devices, this paper proposes a retraining-free embedding compression method that adapts to multiple budget constraints via a single pruning pass. We introduce cooperative game theory and Shapley values—novel in this context—for unbiased, efficient embedding importance estimation, and design a field-aware codebook quantization scheme to enable fine-grained, low-overhead compression. Evaluated on three real-world datasets, our approach achieves recommendation accuracy competitive with state-of-the-art lightweight models, while incurring significantly lower computational and memory overhead. Crucially, it supports dynamic adaptation to varying deployment budgets without retraining. This work establishes a new, practical paradigm for efficient edge-deployable recommendation systems.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression
• Game Theory and Economic Paradigms: Cooperative Game Theory

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Content-based Recommender Systems (CRSs) play a crucial role in shaping user experiences in e-commerce, online advertising, and personalized recommendations. However, due to the vast amount of categorical features, the embedding tables used in CRS models pose a significant storage bottleneck for real-world deployment, especially on resource-constrained devices. To address this problem, various embedding pruning methods have been proposed, but most existing ones require expensive retraining steps for each target parameter budget, leading to enormous computation costs. In reality, this computation cost is a major hurdle in real-world applications with diverse storage requirements, such as federated learning and streaming settings. In this paper, we propose Shapley Value-guided Embedding Reduction (Shaver) as our response. With Shaver, we view the problem from a cooperative game perspective, and quantify each embedding parameter's contribution with Shapley values to facilitate contribution-based parameter pruning. To address the inherently high computation costs of Shapley values, we propose an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we put forward a field-aware codebook to mitigate the information loss in the traditional zero-out treatment. Through extensive experiments on three real-world datasets, Shaver has demonstrated competitive performance with lightweight recommendation models across various parameter budgets. The source code is available at https://github.com/chenxing1999/shaver

Problem

• Content-based Recommendation
• Data Reduction
• System Retraining

Innovation

• Shapley Value
• Data Reduction
• Efficient Recommendation