Consensus Stability of Community Notes on X

2026-01-20

AI Summary

This study investigates the instability of consensus and partial invalidation of Community Notes on platform X following their public display, driven by increasing viewpoint polarization. Leveraging a dataset of 437,396 notes and 35 million user ratings, the authors employ interrupted time series and counterfactual analysis to demonstrate, for the first time, that note exposure significantly alters subsequent rating volume and sentiment, triggering opposing rating behaviors among heterogeneous user groups. The findings reveal that 30.2% of displayed notes lose validity due to post-exposure rating polarization, quantifying the critical role of ideologically divergent users in undermining consensus. This work provides empirical evidence for the dynamic fragility of online fact-checking systems in polarized environments.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Humanities & Computational Social Science
• Reasoning under Uncertainty: Causality

Application Category

• Social Networks and Social Media: Media and governance, opinion dynamics, filter bubbles, polarization
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Community-based fact-checking systems, such as Community Notes on X (formerly Twitter), aim to mitigate online misinformation by surfacing annotations judged helpful by contributors with diverse viewpoints. While prior work has shown that the platform's bridging-based algorithm effectively selects helpful notes at the time of display, little is known about how evaluations change after notes become visible. Using a large-scale dataset of 437,396 community notes and 35 million ratings from over 580,000 contributors, we examine the stability of helpful notes and the rating dynamics that follow their initial display. We find that 30.2% of displayed notes later lose their helpful status and disappear. Using interrupted time series models, we further show that note display triggers a sharp increase in rating volume and a significant shift in rating leaning, but these effects differ across rater groups. Contributors with viewpoints similar to note authors tend to increase supportive ratings, while dissimilar contributors increase negative ratings, producing systematic post-display polarization. Counterfactual analyses suggest that this post-display polarization, particularly from dissimilar raters, plays a substantial role in note disappearance. These findings highlight the vulnerability of consensus-based fact-checking systems to polarized rating behavior and suggest pathways for improving their resilience.

Problem

• consensus stability
• community-based fact-checking
• rating polarization
• misinformation mitigation
• annotation disappearance

Innovation

• consensus stability
• community-based fact-checking
• post-display polarization
• interrupted time series
• rating dynamics