Detecting Effects of AI-Mediated Communication on Language Complexity and Sentiment

2025-04-28

AI Summary

This study investigates the macro-level impact of AI-mediated communication (AI-MC) on linguistic complexity and affective expression in social media discourse. Employing a large-scale longitudinal replication design, we compare structurally aligned Twitter datasets centered on Donald Trump from 2020 (pre-ChatGPT) and 2024, quantifying changes using the Flesch-Kincaid readability index and the VADER sentiment polarity model. Results demonstrate that AI-MC significantly reduces linguistic complexity while inducing a pronounced positive shift in sentiment: mean sentiment polarity increased from 0.04 in 2020 to 0.12 in 2024; neutral content declined by 15 percentage points, and positive expressions rose by 17.3 percentage points. This work constitutes the first rigorous cross-year, same-topic, structured textual replication study to empirically document AI-MC’s systematic reshaping of public discourse style. It provides a critical benchmark for understanding the sociolinguistic effects of generative AI on digital communication.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Application Domains: Humanities & Computational Social Science
• Humans and AI: Learning Human Values and Preferences

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Given the subtle human-like effects of large language models on linguistic patterns, this study examines shifts in language over time to detect the impact of AI-mediated communication (AI- MC) on social media. We compare a replicated dataset of 970,919 tweets from 2020 (pre-ChatGPT) with 20,000 tweets from the same period in 2024, all of which mention Donald Trump during election periods. Using a combination of Flesch-Kincaid readability and polarity scores, we analyze changes in text complexity and sentiment. Our findings reveal a significant increase in mean sentiment polarity (0.12 vs. 0.04) and a shift from predominantly neutral content (54.8% in 2020 to 39.8% in 2024) to more positive expressions (28.6% to 45.9%). These findings suggest not only an increasing presence of AI in social media communication but also its impact on language and emotional expression patterns.

Problem

• Detect AI impact on language complexity in social media
• Analyze sentiment shifts in AI-mediated communication
• Compare pre- and post-ChatGPT linguistic patterns

Innovation

• Comparing pre-ChatGPT and post-ChatGPT tweet datasets
• Analyzing text complexity with Flesch-Kincaid readability
• Measuring sentiment shifts using polarity scores