Learning the effect of persuasion via difference-in-differences

2024-10-18

AI Summary

This paper addresses selection bias in persuasion effect evaluation—specifically, the challenge of distinguishing genuine causal effects on the persuadable from spurious associations among those already convinced or inherently unpersuadable. We propose a novel causal inference framework based on Difference-in-Differences (DID), the first to systematically extend DID for identifying persuasion rates. Unlike conventional approaches, our method requires no exogenous interventions or instrumental variables and is applicable to observational data. It yields more accurate and non-conservative estimates than traditional Average Treatment Effect on the Treated (ATT) estimators, accommodates staggered treatment adoption and event-study designs, and enables dynamic effect analysis. Leveraging regression modeling and semiparametric efficient estimation, we ensure robust inference. Empirical validation using UK general election data and a Chinese curriculum reform initiative successfully quantifies persuasion rates, demonstrating both methodological validity and broad applicability.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

The persuasion rate is a key parameter for measuring the causal effect of a directional message on influencing the recipient's behavior. Its identification has relied on exogenous treatment or the availability of credible instruments, but the requirements are not always satisfied in observational studies. Therefore, we develop a novel econometric framework for the average persuasion rate on the treated and other related parameters by using the difference-in-differences approach. The average treatment effect on the treated is a standard parameter in difference-in-differences, but we show that it is an overly conservative measure in the context of persuasion. For estimation and inference, we propose regression-based approaches as well as semiparametrically efficient estimators. Beginning with the two-period case, we extend the framework to staggered treatment settings, where we show how to conduct richer analyses like the event-study design. We investigate the British election and the Chinese curriculum reform as empirical examples.

Problem

• Measuring persuasive impact of informational treatments on behavior
• Developing causal parameters to refine average treatment effects
• Applying framework to election studies and education reforms

Innovation

• Developed difference-in-differences framework for persuasion measurement
• Introduced forward and backward average persuasion rate parameters
• Proposed regression-based and semiparametrically efficient estimators