From Conversation to Automation: Leveraging Large Language Models to Analyze Strategies in Problem Solving Therapy

2025-01-10

AI Summary

Clinical implementation of Problem-Solving Therapy (PST) suffers from coarse-grained intervention identification and insufficient personalization. Method: We propose the first fine-grained, reusable communication strategy taxonomy and annotation framework for PST dialogues. Leveraging anonymized clinical session transcripts, we design a strategy-level classification pipeline integrating GPT-4o with a fine-tuned Transformer model to automatically identify and classify key therapeutic behaviors—including problem identification, solution generation, decision-making, and evaluation. Results: GPT-4o achieves 76% accuracy on PST strategy identification, significantly outperforming baseline models. Empirical validation confirms that our annotation framework enhances therapists’ real-time intervention precision and system scalability. This work establishes a novel, deployable paradigm for AI-augmented, evidence-based psychological interventions.

Technology Category

• Multiagent Systems: Distributed Problem Solving
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Problem-solving therapy (PST) is a structured psychological approach that helps individuals manage stress and resolve personal issues by guiding them through problem identification, solution brainstorming, decision-making, and outcome evaluation. As mental health care increasingly integrates technologies like chatbots and large language models (LLMs), understanding how PST can be effectively automated is important. This study leverages anonymized therapy transcripts to analyze and classify therapeutic interventions using various LLMs and transformer-based models. Our results show that GPT-4o achieved the highest accuracy (0.76) in identifying PST strategies, outperforming other models. Additionally, we introduced a new dimension of communication strategies that enhances the current PST framework, offering deeper insights into therapist-client interactions. This research demonstrates the potential of LLMs to automate complex therapeutic dialogue analysis, providing a scalable, efficient tool for mental health interventions. Our annotation framework can enhance the accessibility, effectiveness, and personalization of PST, supporting therapists in real-time with more precise, targeted interventions.

Problem

• Computer Models
• Problem-Solving Therapy
• Personalization

Innovation

• GPT-4o
• Problem-Solving Therapy
• Enhanced Communication Skills