The Illusion of Empathy: How AI Chatbots Shape Conversation Perception

2024-11-19

arXiv.org

AI Summary

This study investigates how users’ perception of AI chatbots’ empathy influences conversational quality—a critical yet underexplored gap in human-AI interaction research. Method: Analyzing 155 human-AI dialogues, we integrate user-reported quality ratings, GPT-4o–based multi-dimensional automated empathy annotation, and cross-dataset comparative analysis. Contribution/Results: Despite GPT-based models significantly outperforming humans in objective dialogue quality metrics, users consistently perceive their empathy as systematically lower—empirically revealing, for the first time, the “high-quality dialogue ≠ high-empathy perception” misalignment. Crucially, merely embedding empathic language does not enhance user experience; instead, bridging the interpretive gap between AI-generated empathy cues and user expectations is essential. These findings challenge the prevailing design paradigm equating empathic language with empathic experience, offering both theoretical grounding and methodological guidance for developing genuinely high-quality empathic human-AI interactions.

Technology Category

• Humans and AI: User Experience and Usability
• Natural Language Processing: Conversational AI/Dialog Systems
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

As AI chatbots increasingly incorporate empathy, understanding user-centered perceptions of chatbot empathy and its impact on conversation quality remains essential yet under-explored. This study examines how chatbot identity and perceived empathy influence users' overall conversation experience. Analyzing 155 conversations from two datasets, we found that while GPT-based chatbots were rated significantly higher in conversational quality, they were consistently perceived as less empathetic than human conversational partners. Empathy ratings from GPT-4o annotations aligned with user ratings, reinforcing the perception of lower empathy in chatbots compared to humans. Our findings underscore the critical role of perceived empathy in shaping conversation quality, revealing that achieving high-quality human-AI interactions requires more than simply embedding empathetic language; it necessitates addressing the nuanced ways users interpret and experience empathy in conversations with chatbots.

Problem

• AI chatbot empathy perception
• Conversation quality evaluation
• Human-AI interaction nuances

Innovation

• GPT-based chatbot analysis
• Empathy perception comparison
• Conversation quality evaluation