Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings

2026-06-02

AI Summary

This study addresses the challenge of identifying the causal effects of individual attributes on overall ratings in online reviews, particularly when attributes exhibit strong correlations. Building upon the CausalBERT framework, the authors treat textual mentions as proxy variables for latent attributes and enhance treatment assignment calibration through temperature scaling. They further refine hyperparameters to mitigate over-adjustment for confounders and integrate interpretability techniques to uncover potential hidden confounding factors. Empirical evaluation on a dataset of 600,000 U.S. K–12 school reviews demonstrates that the proposed approach substantially improves the reliability of causal effect estimation, revealing school administration and baseline academic performance as key drivers of overall ratings.

Abstract

Online reviews provide valuable insights into the perceived quality of facets of a product or service. While aspect-based sentiment analysis has focused on extracting these facets from reviews, there is less work understanding the impact of each aspect on overall perception. This is particularly challenging given correlations among aspects, making it difficult to isolate the effects of each. This paper introduces a methodology based on recent advances in text-based causal analysis, specifically CausalBERT, to disentangle the effect of each factor on overall review ratings. We enhance CausalBERT with three key improvements: temperature scaling for better calibrated treatment assignment estimates; hyperparameter optimization to reduce confound overadjustment; and interpretability methods to characterize discovered confounds. In this work, we treat the textual mentions in reviews as proxies for real-world attributes. We validate our approach on real and semi-synthetic data from over 600K reviews of U.S. K-12 schools. We find that the proposed enhancements result in more reliable estimates, and that perception of school administration and performance on benchmarks are significant drivers of overall school ratings.

Problem

• online reviews
• causal inference
• aspect influence
• rating disentanglement
• confounding factors

Innovation

• text-based causal inference
• CausalBERT
• temperature scaling
• confound adjustment
• aspect disentanglement