Inherit4Rec: Parameter Inheritance for Efficient Scaling of Recommendation Models

2026-09-19

AI Summary

为解决推荐系统中模型扩展的效率问题，提出Inherit4Rec框架，通过参数继承支持密集到密集的增长和密集到稀疏的转换。

Abstract

Scaling model capacity has emerged as an effective approach to overcoming performance bottlenecks in industrial recommender systems. However, repeatedly training larger dense models from scratch demands substantial data and time, while their growing computation conflicts with the strict serving budgets of industrial systems. Parameter inheritance provides a promising route for both dense model growth and sparse conversion, yet existing methods are primarily designed for static corpora and can suffer sharp performance drops under dynamically evolving recommendation data. To address these challenges, we propose Inherit4Rec, a parameter-inheritance framework that supports both Dense-to-Dense (D2D) growth and Dense-to-Sparse (D2S) conversion. Inherit4Rec-D2D combines hybrid growth with asymmetric training to preserve the forward function at expansion and maintain update continuity. Inherit4Rec-D2S constructs SMoE networks through co-activation-aware partitioning and a load-balancing loss, preserving dense-model capabilities while promoting balanced expert activation. Experiments on KuaiRand-1K and an industrial short-video recommendation dataset show that both transformations consistently outperform the evaluated inheritance baselines across all prediction objectives. These results demonstrate the effectiveness of Inherit4Rec for continual capacity expansion and computation-efficient sparse conversion in industrial recommender systems.

Problem

• model capacity
• recommender systems
• parameter inheritance
• dense models
• dynamic data

Innovation

• Parameter Inheritance
• Dense-to-Dense Growth
• Dense-to-Sparse Conversion
• SMoE Networks