Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy

2024-11-11

arXiv.org

AI Summary

Large language models (LLMs) in psychotherapy chatbots suffer from weak controllability, low therapeutic adherence, and excessive reliance on handcrafted scripts. Method: We propose Script-Strategy Aligned Generation (SSAG), a novel paradigm that aligns LLM outputs with clinical strategies—e.g., cognitive behavioral therapy (CBT) frameworks—during dynamic generation, rather than merely matching static scripts. SSAG integrates instruction-tuned prompting, supervised fine-tuning, and strategy-guided multi-stage alignment to yield an interpretable and controllable generation mechanism. Contribution/Results: In a 10-day field study, SSAG achieved clinically viable performance in therapeutic adherence, user trust, and intervention efficacy—matching fully scripted systems and significantly outperforming rule-based baselines. This work establishes a new pathway for digital mental health interventions that balances clinical rigor, safety, and development efficiency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Multiagent Systems: Adversarial Agents

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Chatbots or conversational agents (CAs) are increasingly used to improve access to digital psychotherapy. Many current systems rely on rigid, rule-based designs, heavily dependent on expert-crafted dialogue scripts for guiding therapeutic conversations. Although recent advances in large language models (LLMs) offer the potential for more flexible interactions, their lack of controllability and transparency poses significant challenges in sensitive areas like psychotherapy. In this work, we explored how aligning LLMs with expert-crafted scripts can enhance psychotherapeutic chatbot performance. Our comparative study showed that LLMs aligned with expert-crafted scripts through prompting and fine-tuning significantly outperformed both pure LLMs and rule-based chatbots, achieving a more effective balance between dialogue flexibility and adherence to therapeutic principles. Building on findings, we proposed ``Script-Strategy Aligned Generation (SSAG)'', a flexible alignment approach that reduces reliance on fully scripted content while enhancing LLMs' therapeutic adherence and controllability. In a 10-day field study, SSAG demonstrated performance comparable to full script alignment and outperformed rule-based chatbots, empirically supporting SSAG as an efficient approach for aligning LLMs with domain expertise. Our work advances LLM applications in psychotherapy by providing a controllable, adaptable, and scalable solution for digital interventions, reducing reliance on expert effort. It also provides a collaborative framework for domain experts and developers to efficiently build expertise-aligned chatbots, broadening access to psychotherapy and behavioral interventions.

Problem

• Aligning LLMs with expert-crafted psychotherapy scripts for controllability
• Reducing reliance on rigid rule-based systems in therapeutic chatbots
• Enhancing LLM adherence to therapeutic principles while maintaining flexibility

Innovation

• Aligning LLMs with expert-crafted psychotherapy scripts
• Combining fine-tuning and prompting for therapeutic adherence
• Reducing script reliance while maintaining controllability