Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration

2026-07-26

AI Summary

This study addresses the challenge that large language models struggle to generate empathetic, culturally appropriate, and ethically compliant responses in mental health counseling for low-resource languages. To tackle this issue, the authors propose a Role-Playing Reflexive Chain-of-Thought (RP-RCAF) prompting framework and introduce G-REFS, a multidimensional evaluation system integrating real-world case studies and expert guidance. The approach combines few-shot prompting, role-playing, and a structured self-reflection mechanism, with effectiveness validated through both automated metrics and expert evaluations. Experimental results demonstrate that RP-RCAF significantly outperforms existing prompting strategies, producing responses that closely approximate professional counselors in emotional sensitivity, cultural appropriateness, linguistic clarity, and ethical compliance.

Abstract

Despite recent advances in large language models (LLMs), their ability to generate empathetic mental health counseling responses in low-resource languages remains largely unexplored. To address this gap, we curate 625 authentic mental health cases from three complementary sources: (1) publicly available Facebook posts discussing mental health concerns, (2) transcripts from the Bangladeshi television program "Ami Akhon Ki Korbo", and (3) anonymized student questionnaire responses covering diverse emotional and psychological challenges. Based on these cases, we build an evaluation corpus comprising advice written by licensed clinical psychologists and responses generated by three modern proprietary LLMs: GPT-4o Mini, Claude 4.5 Haiku, and Gemini 2.5 Pro. We further propose the Role-Playing Reflective Chain-of-Thought Advisory Framework (RP-RCAF), a task-specific prompting strategy that combines expert-authored few-shot examples with structured self-reflection to produce supportive, culturally aware, and ethically aligned counseling through a compassionate advisor persona. We also introduce the Grok 4-Based Response Evaluation and Scoring Framework (G-REFS), which integrates automated assessment with expert psychologist validation across emotional sensitivity, cultural appropriateness, linguistic clarity, and ethical soundness. Experimental results show that RP-RCAF consistently outperforms conventional prompting across all evaluated models and produces responses that more closely align with professional psychological counseling.

Problem

• empathy
• mental health counseling
• low-resource languages
• cultural sensitivity
• language models

Innovation

• Role-Playing Reflective Chain-of-Thought
• Culturally Sensitive Mental Health Advice
• Human-LLM Collaboration
• Response Evaluation Framework
• Low-Resource Language Counseling