User and Recommender Behavior Over Time: Contextualizing Activity, Effectiveness, Diversity, and Fairness in Book Recommendation

2025-05-07

AI Summary

This work investigates the long-term temporal evolution of data and behavioral patterns in recommender systems, focusing on dynamics in user engagement, recommendation diversity, and fairness—particularly assessing the causal impact of Goodreads’ 2011 algorithmic recommendation rollout. Leveraging the complete 2006–2023 book interaction dataset, we construct the first temporal-evolution-aware multi-dimensional evaluation framework, integrating rolling-window collaborative filtering (SVD++, BPR, LightGCN) with time-sensitive metrics—including Gini coefficient, entropy-based diversity, and exposure fairness. Our analysis uncovers a novel “evolutionary bias”: persistent decline in diversity, increasing exposure inequality favoring head items, and marked deterioration in fairness for new users. We empirically confirm that algorithmic recommendations amplify the Matthew effect, challenging prevailing static temporal-split evaluation paradigms and underscoring the necessity of evolutionary assessment in real-world recommendation deployment.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Data is an essential resource for studying recommender systems. While there has been significant work on improving and evaluating state-of-the-art models and measuring various properties of recommender system outputs, less attention has been given to the data itself, particularly how data has changed over time. Such documentation and analysis provide guidance and context for designing and evaluating recommender systems, particularly for evaluation designs making use of time (e.g., temporal splitting). In this paper, we present a temporal explanatory analysis of the UCSD Book Graph dataset scraped from Goodreads, a social reading and recommendation platform active since 2006. We measure the book interaction data using a set of activity, diversity, and fairness metrics; we then train a set of collaborative filtering algorithms on rolling training windows to observe how the same measures evolve over time in the recommendations. Additionally, we explore whether the introduction of algorithmic recommendations in 2011 was followed by observable changes in user or recommender system behavior.

Problem

• Analyze temporal changes in book recommendation data metrics
• Evaluate impact of algorithmic recommendations on user behavior
• Measure activity, diversity, and fairness in recommender systems

Innovation

• Temporal analysis of Goodreads book interaction data
• Collaborative filtering on rolling training windows
• Measuring activity, diversity, fairness metrics over time