PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurements

2024-07-22

arXiv.org

AI Summary

To address the limitations of conventional psychological assessments—namely, their lack of personalization, inflexibility, and low accessibility—this paper proposes the first progressive deep dialogue assessment framework tailored for psychometrics. Using the Myers-Briggs Type Indicator (MBTI) as a representative case, the framework integrates large language models, multimodal user modeling, and optimized multi-turn dialogue strategies to enable contextually coherent, personalized interactions. It introduces a novel surrogate-variable-based learning-to-rank training paradigm, facilitating incremental personality type classification under a unified, interpretable scoring model. Experimental results demonstrate statistically significant improvements over single-turn QA baselines on MBTI classification. Deployed online, the system attracted over 3,000 unique visitors and collected high-quality, expert-annotated multi-turn dialogue data. The implementation is publicly released via OpenDILab to foster reproducibility and community advancement.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In the field of psychology, traditional assessment methods, such as standardized scales, are frequently critiqued for their static nature, lack of personalization, and reduced participant engagement, while comprehensive counseling evaluations are often inaccessible. The complexity of quantifying psychological traits further limits these methods. Despite advances with large language models (LLMs), many still depend on single-round Question-and-Answer interactions. To bridge this gap, we introduce PsyDI, a personalized and progressively in-depth chatbot designed for psychological measurements, exemplified by its application in the Myers-Briggs Type Indicator (MBTI) framework. PsyDI leverages user-related multi-modal information and engages in customized, multi-turn interactions to provide personalized, easily accessible measurements, while ensuring precise MBTI type determination. To address the challenge of unquantifiable psychological traits, we introduce a novel training paradigm that involves learning the ranking of proxy variables associated with these traits, culminating in a robust score model for MBTI measurements. The score model enables PsyDI to conduct comprehensive and precise measurements through multi-turn interactions within a unified estimation context. Through various experiments, we validate the efficacy of both the score model and the PsyDI pipeline, demonstrating its potential to serve as a general framework for psychological measurements. Furthermore, the online deployment of PsyDI has garnered substantial user engagement, with over 3,000 visits, resulting in the collection of numerous multi-turn dialogues annotated with MBTI types, which facilitates further research. The source code for the training and web service components is publicly available as a part of OpenDILab at: https://github.com/opendilab/PsyDI

Problem

• Personalized Psychological Assessment
• Chatbot Development
• Large-scale Language Model

Innovation

• PsyDI Chatbot
• MBTI Measurement
• Personalized Dialogue