AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior

2025-03-03

AI Summary

This study investigates the real-world efficacy of large language models (LLMs) in promoting pro-environmental behavior (PEB), specifically asking whether LLM-driven persuasive interventions yield systematically different effects across real humans, empirically calibrated simulated agents, and fully synthetic agents. Method: Integrating four evidence-based persuasion frameworks—including Moral Foundations Theory—the authors deployed LLMs to generate personalized intervention dialogues. A controlled experiment was conducted with 3,200 human participants alongside high-fidelity simulated and synthetic agents. Contribution/Results: The study identifies the “synthetic persuasion paradox”: LLM interventions significantly improved PEB attitudes among simulated and synthetic agents (p < 0.001) but showed no statistically significant effect on real humans. Simulated agents exhibited response patterns qualitatively similar to humans yet consistently overestimated intervention efficacy. These findings expose critical limitations of LLMs in real-world environmental behavior change, establish synthetic agents as valuable for preliminary intervention screening—but not as substitutes for longitudinal human trials—and define methodological boundaries for AI-driven sustainability research.

Technology Category

Cognitive Modeling & Cognitive Systems: Simulating Human BehaviorMultiagent Systems: Agent-Based Simulation and Emergent BehaviorMachine Learning: Large Multimodal Models (LMMs)

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive. We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants: real humans (n=1,200), simulated humans based on actual participant data (n=1,200), and fully synthetic personas (n=1,200). All three participant groups faced personalized or standard chatbots, or static statements, employing four persuasion strategies (moral foundations, future self-continuity, action orientation, or"freestyle"chosen by the LLM). Results reveal a"synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift. Simulated participants better approximate human trends but still overestimate effects. This disconnect underscores LLM's potential for pre-evaluating PEB interventions but warns of its limits in predicting real-world behavior. We call for refined synthetic modeling and sustained and extended human trials to align conversational AI's promise with tangible sustainability outcomes.

Problem

• Evaluating LLMs' effectiveness in promoting pro-environmental behavior.
• Comparing persuasion impacts on humans, simulated humans, and synthetic personas.
• Highlighting the gap between synthetic and real-world behavioral predictions.

Innovation

• LLMs used to promote pro-environmental behavior
• Comparison of real, simulated, and synthetic participants
• Synthetic persuasion paradox revealed in results