A Robot That Listens: Enhancing Self-Disclosure and Engagement Through Sentiment-based Backchannels and Active Listening

2025-09-09

AI Summary

This study addresses the limited socio-emotional listening capability of social robots by proposing a dynamic response framework that integrates real-time emotion recognition with active listening strategies. Methodologically, it introduces the first end-to-end system that jointly leverages speech-based real-time sentiment analysis, multimodal active listening behaviors (e.g., semantic paraphrasing, backchanneling utterances, and head nods), and large language model–driven, context-aware generative responses. Its key contribution is a sentiment-driven listening闭环 (closed loop) enabling coordinated verbal and nonverbal feedback grounded in user affect. User studies demonstrate statistically significant improvements over baseline systems: deeper self-disclosure (p < 0.01), higher dialogue satisfaction (+32.7%), and stronger perceived empathy (Cohen’s d = 0.94), confirming that affect-adaptive active listening meaningfully enhances human–robot relational quality.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As social robots get more deeply integrated intoour everyday lives, they will be expected to engage in meaningful conversations and exhibit socio-emotionally intelligent listening behaviors when interacting with people. Active listening and backchanneling could be one way to enhance robots' communicative capabilities and enhance their effectiveness in eliciting deeper self-disclosure, providing a sense of empathy,and forming positive rapport and relationships with people.Thus, we developed an LLM-powered social robot that can exhibit contextually appropriate sentiment-based backchannelingand active listening behaviors (active listening+backchanneling) and compared its efficacy in eliciting people's self-disclosurein comparison to robots that do not exhibit any of these listening behaviors (control) and a robot that only exhibitsbackchanneling behavior (backchanneling-only). Through ourexperimental study with sixty-five participants, we found theparticipants who conversed with the active listening robot per-ceived the interactions more positively, in which they exhibited the highest self-disclosures, and reported the strongest senseof being listened to. The results of our study suggest that the implementation of active listening behaviors in social robotshas the potential to improve human-robot communication andcould further contribute to the building of deeper human-robot relationships and rapport.

Problem

• Enhancing robot self-disclosure and engagement through listening behaviors
• Developing sentiment-based backchanneling for social robots
• Improving human-robot communication and relationship building

Innovation

• LLM-powered robot with sentiment-based backchannels
• Active listening behaviors to enhance engagement
• Contextual responses to improve self-disclosure