It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation

2026-07-20

AI Summary

This study addresses the limitations of current AI-assisted information evaluation systems, which predominantly rely on directive rhetoric that suppresses users’ active critical thinking and lack systematic investigation into the effects of diverse rhetorical strategies. The work proposes and systematically designs eight thought-provoking rhetorical modes—including Socratic questioning, scaffolding explanations, and adversarial misinformation—and evaluates their impact through a within-subject human-AI interaction experiment in an on-demand fact-checking task. Findings reveal that scaffolding explanations significantly enhance judgment accuracy and foster deeper reflection, while adversarial conditions yield modest accuracy gains. Users expressed the strongest preference for alternative framing and the least for explanatory alternatives, primarily due to perceived higher cognitive load. The study underscores the pivotal role of rhetorical form in shaping users’ cognitive engagement and decision-making.

Abstract

Prior work on AI-assisted information evaluation has largely focused on what AI systems communicate, comparing explanation types and formats, with responses predominantly cast in directive rhetoric where the system delivers a verdict and the user passively accepts it. While debate-style interactions have recently shown promise in prompting critical evaluation over deference, the rhetorical patterns that structure AI responses and how they might induce reflection, uncertainty, or independent reasoning remain largely unexamined. To address this, we investigated eight rhetorical patterns known to induce contemplation: Intentional Misleading, Interpretive Alternative, Scaffold Explanation, Triggering Distrust, Information Distortion, Alternative Framing, Socratic Questioning, and an Oracle baseline. Through a within-subject study with n=98 participants on a hint-on-demand fact verification task, we observed preliminary evidence that Scaffold Explanation were associated with the highest accuracy gains, and encouraging deeper reflection. Surprisingly, the adversarial conditions also improved accuracy modestly. Participants preferred Alternative Framing most and Interpretive Alternative least, largely due to the latter's perceived time cost. We discuss the implications of designing conversational agents with varied rhetorical styles and the trade-offs among user performance, satisfaction, and contemplation.

Problem

• AI-assisted information evaluation
• rhetorical patterns
• critical evaluation
• conversational agents
• user reasoning

Innovation

• rhetorical patterns
• AI-assisted evaluation
• Scaffold Explanation
• critical reflection
• conversational agents