Measuring Social Media Polarization Using Large Language Models and Heuristic Rules

2026-01-02

arXiv.org

AI Summary

This work proposes a novel approach that integrates large language models with interpretable heuristic rules to quantify affective polarization in social media discourse surrounding contentious issues such as climate change and gun control. By jointly modeling user stance, sentiment orientation, and interaction dynamics, the method constructs a rule-based scoring system that enables fine-grained and scalable polarization measurement—from individual exchanges to large-scale conversational contexts. The study identifies two distinct event-driven polarization patterns for the first time: “anticipatory” and “reactive” types—each associated with observable polarization peaks before and after critical events, respectively. The framework demonstrates strong generalizability while maintaining high interpretability, offering a transparent and adaptable tool for analyzing affective polarization in dynamic online environments.

Technology Category

• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Machine Learning: Learning Preferences or Rankings

Application Category

• Social Networks and Social Media: Media and governance, opinion dynamics, filter bubbles, polarization
• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Understanding affective polarization in online discourse is crucial for evaluating the societal impact of social media interactions. This study presents a novel framework that leverages large language models (LLMs) and domain-informed heuristics to systematically analyze and quantify affective polarization in discussions on divisive topics such as climate change and gun control. Unlike most prior approaches that relied on sentiment analysis or predefined classifiers, our method integrates LLMs to extract stance, affective tone, and agreement patterns from large-scale social media discussions. We then apply a rule-based scoring system capable of quantifying affective polarization even in small conversations consisting of single interactions, based on stance alignment, emotional content, and interaction dynamics. Our analysis reveals distinct polarization patterns that are event dependent: (i) anticipation-driven polarization, where extreme polarization escalates before well-publicized events, and (ii) reactive polarization, where intense affective polarization spikes immediately after sudden, high-impact events. By combining AI-driven content annotation with domain-informed scoring, our framework offers a scalable and interpretable approach to measuring affective polarization. The source code is publicly available at: https://github.com/hasanjawad001/llm-social-media-polarization.

Problem

• affective polarization
• social media
• large language models
• stance alignment
• emotional content

Innovation

• large language models
• affective polarization
• heuristic rules
• stance extraction
• social media analysis