🤖 AI Summary
This paper investigates the choice of training objectives in e-commerce recommendation systems: click-through rate (CTR) versus order submission rate (OSR). Using large-scale online A/B tests on an industrial multi-objective recommendation model, we jointly analyze click, add-to-cart, and purchase behavioral data to systematically evaluate the impact of CTR- versus OSR-oriented optimization on GMV, new-item exposure, and user exploration diversity. Results demonstrate that optimizing for OSR increases GMV by over fivefold compared to CTR, without compromising new-item visibility or user behavioral diversity. Feature importance analysis further reveals a fundamental shift in model attention toward conversion-critical signals under OSR optimization. The study provides empirical evidence and methodological guidance for objective design in e-commerce recommender systems, highlighting the superiority of downstream conversion metrics over proximal engagement proxies in driving business outcomes.
📝 Abstract
Ranking product recommendations to optimize for a high click-through rate (CTR) or for high conversion, such as add-to-cart rate (ACR) and Order-Submit-Rate (OSR, view-to-purchase conversion) are standard practices in e-commerce. Optimizing for CTR appears like a straightforward choice: Training data (i.e., click data) are simple to collect and often available in large quantities. Additionally, CTR is used far beyond e-commerce, making it a generalist, easily implemented option. ACR and OSR, on the other hand, are more directly linked to a shop's business goals, such as the Gross Merchandise Value (GMV). In this paper, we compare the effects of using either of these objectives using an online A/B test. Among our key findings, we demonstrate that in our shops, optimizing for OSR produces a GMV uplift more than five times larger than when optimizing for CTR, without sacrificing new product discovery. Our results also provide insights into the different feature importances for each of the objectives.