Item Level Exploration Traffic Allocation in Large-scale Recommendation Systems

2025-05-14

AI Summary

This paper addresses the cold-start problem for new items in large-scale recommender systems by proposing a dynamic exploration mechanism grounded in discoverability modeling. Methodologically, it introduces (1) a learnable probabilistic discoverability model that quantifies the exposure potential of new items under the current traffic allocation policy, and (2) an adaptive exploration budget allocation algorithm that optimizes online traffic scheduling to balance short-term exposure efficiency and long-term ecosystem diversity. The approach integrates probabilistic modeling, online traffic control, and distributed decision optimization. Evaluated on real-world industrial-scale deployments, it significantly improves both the first-day exposure rate and 30-day retention rate of new items, while broadening item catalog coverage and enriching long-tail content diversity. Empirical results demonstrate its effectiveness and scalability under high-concurrency, low-latency production constraints.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking

Abstract

This paper contributes to addressing the item cold start problem in large-scale recommender systems, focusing on how to efficiently gain initial visibility for newly ingested content. We propose an exploration system designed to efficiently allocate impressions to these fresh items. Our approach leverages a learned probabilistic model to predict an item's discoverability, which then informs a scalable and adaptive traffic allocation strategy. This system intelligently distributes exploration budgets, optimizing for the long-term benefit of the recommendation platform. The impact is a demonstrably more efficient cold-start process, leading to a significant increase in the discoverability of new content and ultimately enriching the item corpus available for exploitation, as evidenced by its successful deployment in a large-scale production environment.

Problem

• Address item cold start in large recommender systems
• Efficiently allocate impressions to new items
• Optimize long-term discoverability of fresh content

Innovation

• Learned probabilistic model predicts item discoverability
• Scalable adaptive traffic allocation strategy
• Intelligent exploration budget distribution for long-term benefit