Persuasion at Play: Understanding Misinformation Dynamics in Demographic-Aware Human-LLM Interactions

2025-03-03

AI Summary

This study investigates the bidirectional persuasive dynamics between humans and large language models (LLMs) in misinformation propagation, with a focus on how demographic attributes—age, gender, and education—affect susceptibility. We propose the first multi-agent LLM simulation framework explicitly integrating real human stance data and embedding demographic variables into agent behavioral modeling. Experiments reveal that LLMs exhibit demographic sensitivity patterns consistent with empirical human responses; multi-agent systems spontaneously form human-like echo chambers without external intervention; and LLM-generated persuasive content reinforces preexisting biases. These findings provide a quantifiable theoretical foundation and empirical evidence for identifying high-risk demographic subgroups and designing targeted, adaptive interventions against misinformation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Existing challenges in misinformation exposure and susceptibility vary across demographic groups, as some populations are more vulnerable to misinformation than others. Large language models (LLMs) introduce new dimensions to these challenges through their ability to generate persuasive content at scale and reinforcing existing biases. This study investigates the bidirectional persuasion dynamics between LLMs and humans when exposed to misinformative content. We analyze human-to-LLM influence using human-stance datasets and assess LLM-to-human influence by generating LLM-based persuasive arguments. Additionally, we use a multi-agent LLM framework to analyze the spread of misinformation under persuasion among demographic-oriented LLM agents. Our findings show that demographic factors influence susceptibility to misinformation in LLMs, closely reflecting the demographic-based patterns seen in human susceptibility. We also find that, similar to human demographic groups, multi-agent LLMs exhibit echo chamber behavior. This research explores the interplay between humans and LLMs, highlighting demographic differences in the context of misinformation and offering insights for future interventions.

Problem

• Understanding misinformation dynamics in human-LLM interactions.
• Analyzing demographic influence on susceptibility to misinformation.
• Exploring echo chamber behavior in multi-agent LLM frameworks.

Innovation

• Analyzes human-LLM bidirectional persuasion dynamics
• Uses multi-agent LLM framework for misinformation spread
• Generates LLM-based persuasive arguments for influence assessment