The Value of Personalized Recommendations: Evidence from Netflix

2025-11-10

Social Science Research Network

AI Summary

This paper disentangles the *algorithmic value* of personalized recommendation systems (e.g., Netflix) from the *intrinsic value* of content itself, and quantifies their causal effects on user engagement and consumption diversity. We propose a discrete choice model integrating recommendation utility, low-rank heterogeneity, and state dependence, leveraging exogenous variation induced by the algorithm to identify causal impacts. To distinguish effective recommendations from mechanical exposure, we innovatively design a model-free split-ratio validation method. Estimation combines matrix factorization, low-rank estimation, and counterfactual analysis on large-scale real-world viewing data. Results show that current recommender systems increase engagement by 4% over a matrix factorization baseline and by 12% over a popularity baseline. Critically, they significantly broaden consumption toward moderately popular content—demonstrating their pivotal role in balancing recommendation efficiency with content diversity.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Algorithm Configuration

Application Category

• User Modeling, Personalization and Recommendation: Personalized content ranking and presentation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Personalized recommendation systems shape much of user choice online, yet their targeted nature makes separating out the value of recommendation and the underlying goods challenging. We build a discrete choice model that embeds recommendation-induced utility, low-rank heterogeneity, and flexible state dependence and apply the model to viewership data at Netflix. We exploit idiosyncratic variation introduced by the recommendation algorithm to identify and separately value these components as well as to recover model-free diversion ratios that we can use to validate our structural model. We use the model to evaluate counterfactuals that quantify the incremental engagement generated by personalized recommendations. First, we show that replacing the current recommender system with a matrix factorization or popularity-based algorithm would lead to 4% and 12% reduction in engagement, respectively, and decreased consumption diversity. Second, most of the consumption increase from recommendations comes from effective targeting, not mechanical exposure, with the largest gains for mid-popularity goods (as opposed to broadly appealing or very niche goods).

Problem

• Measuring the value of personalized recommendations versus inherent content quality
• Quantifying engagement impact of alternative recommendation algorithms on Netflix
• Analyzing how recommendation targeting versus exposure drives consumption patterns

Innovation

• Discrete choice model with recommendation-induced utility
• Exploits algorithm variation to identify recommendation value
• Quantifies engagement gains from personalized targeting