🤖 AI Summary
This work addresses the challenge of accurately estimating the causal effect of content features on user engagement in personalized recommendation systems, where observational data are confounded by unmeasured factors and high-dimensional content spaces. The authors propose a novel method that leverages the randomness inherent in content display algorithms to identify causal effects using only a single non-displayed item per user log and the ratio of display probabilities between displayed and non-displayed items. Built upon counterfactual reasoning and importance weighting, the approach yields an unbiased estimator without requiring parametric assumptions about the content space or user preferences. The paper establishes theoretical guarantees for causal identifiability and introduces a new estimator enabling robust inference, thereby providing a reliable data-driven foundation for optimizing personalized content delivery.
📝 Abstract
Recommendation systems and chatbots present content to users, typically using stochastic algorithms that select the content based on user characteristics or context. Examples of content include chat responses, videos, or items available for purchase. Scientists and application developers are often interested in whether characteristics of content increase outcomes such as user engagement. Estimates of such causal effects may guide content providers to generate content that emphasize desirable features. However, in settings with a large content library or where content is generated uniquely for a given user, it can be difficult to use observational data to learn the causal effect of content features, because the content a user sees is tailored to that user, and because content varies in many dimensions. This paper proposes a new method for estimating the impact of content features using observational data, when the algorithm that determines user exposure incorporates some randomization, and when two additional data elements are logged for each user: $(i)$ the identity of at least one item that could have been exposed to the user, but was not (the unexposed item); $(ii)$ an estimate of the ratio of the probability that the unexposed item would have been shown to the probability that the exposed item was shown. We show that causal effects of features are identified in this setting, even in the presence of unobserved confounders that affect both user preferences and the identity of the considered pair of items (exposed and unexposed). Our estimator differs from prior approaches in terms of what data is used and how the estimator is constructed.