Therapeutic AI and the Hidden Risks of Over-Disclosure: An Embedded AI-Literacy Framework for Mental Health Privacy

2025-10-12

AI Summary

Large language models (LLMs) are increasingly deployed in mental health applications, yet the absence of standardized data governance frameworks risks eliciting excessive disclosure of irrelevant sensitive information from users—exacerbating privacy violations, model bias, and data misuse. To address this, we propose the first intervention framework that embeds AI literacy education *in situ* within psychological dialogue systems. Integrating LLMs, human-computer interaction design, and behavioral intervention strategies, our approach employs dynamic prompting, real-time risk feedback, and controllable disclosure guidance to enhance users’ awareness of and agency over data collection, storage, and reuse. Experimental evaluation demonstrates that the framework significantly reduces inappropriate information disclosure, improves user trust, and enhances overall experience. It establishes an evaluable, scalable privacy-enhancing paradigm for deploying LLMs safely in sensitive health domains.

Technology Category

• Machine Learning: Privacy
• Humans and AI: Intelligent User Interfaces
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

Large Language Models (LLMs) are increasingly deployed in mental health contexts, from structured therapeutic support tools to informal chat-based well-being assistants. While these systems increase accessibility, scalability, and personalization, their integration into mental health care brings privacy and safety challenges that have not been well-examined. Unlike traditional clinical interactions, LLM-mediated therapy often lacks a clear structure for what information is collected, how it is processed, and how it is stored or reused. Users without clinical guidance may over-disclose personal information, which is sometimes irrelevant to their presenting concern, due to misplaced trust, lack of awareness of data risks, or the conversational design of the system. This overexposure raises privacy concerns and also increases the potential for LLM bias, misinterpretation, and long-term data misuse. We propose a framework embedding Artificial Intelligence (AI) literacy interventions directly into mental health conversational systems, and outline a study plan to evaluate their impact on disclosure safety, trust, and user experience.

Problem

• LLMs in mental health lack clear data collection and processing structures
• Users may over-disclose personal information without clinical guidance
• Overexposure raises privacy concerns and potential for data misuse

Innovation

• Embedding AI literacy into mental health systems
• Evaluating impact on disclosure safety and trust
• Addressing over-disclosure risks in LLM therapy