A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

2026-06-03

AI Summary

Traditional persuasion research relies solely on pre- and post-persuasion belief measurements, failing to capture the dynamic evolution of beliefs during dialogue. This work proposes PERSUASIONTRACE, a framework that systematically models persuasion in human–large language model interactions through multi-turn belief tracking, annotation of rhetorical strategies (logos, pathos, ethos), and simulator-based evaluation. Its core innovation lies in a Bayesian network–based simulated target model that explicitly maintains latent belief states evolving over time, enabling cognitively plausible belief updating and shifting persuasion assessment from outcome-oriented metrics toward process fidelity. Experiments reveal two distinct human belief-updating patterns sensitive to rhetorical strategies; the proposed model achieves a human similarity score of 81 (human baseline: 80), significantly outperforming baseline LLMs (64).

Abstract

Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint measures identify whether persuasion occurred, yet miss where and how beliefs moved within a dialogue. We present PERSUASIONTRACE, a framework for studying persuasion in human-LLM interaction. Built on a web-based experimental platform, PERSUASIONTRACE contributes a tool for multi-turn persuasion studies and a process-level evaluation protocol: it records multi-turn belief reports from human or simulated targets of persuasion, annotates persuader turns with rhetorical dimensions (logos/pathos/ethos), and evaluates simulators by fidelity to real human belief dynamics. Using this framework, we find that human targets group into two clusters of multi-turn belief updates and exhibit susceptibility to rhetorical strategies, and that LLMs are persuasive across generic and personalized topics, text and audio modalities, and multi-turn interactions. Prior work has chiefly used vanilla-prompted LLMs to simulate human targets, but we show that these simulators fail to replicate human belief dynamics. We introduce a Bayesian-network simulated target that maintains an explicit latent belief state over time so each persuader message yields cognitively realistic belief updates. In human-likeness evaluation, our Bayesian target scores near a human reference (81 vs 80), while baseline LLM targets score substantially lower (64). PERSUASIONTRACE reframes persuasion evaluation from endpoint movement alone to process fidelity, providing a stronger basis for scientific analysis and safer optimization of persuasive systems.

Problem

• persuasion
• belief dynamics
• multi-turn dialogue
• human-LLM interaction
• process-level evaluation

Innovation

• multi-turn persuasion
• belief tracing
• Bayesian network
• process-level evaluation
• LLM-human interaction