🤖 AI Summary
Current large language models (LLMs) lack systematic evaluation of interpretive reliability in psychiatric clinical risk assessment and are susceptible to interference from non-clinical information and prompt design. This study introduces the first LLM reliability auditing framework tailored to psychiatry, systematically evaluating the stability of hospitalization risk scores across four leading models—Gemini, LLaMA, Claude, and GPT—using synthetically generated patient profiles that incorporate both clinical and non-clinical features, alongside four prompting strategies: neutral, logical, humanistic influence, and clinical judgment. Results demonstrate that non-clinical information significantly increases both the mean risk scores and output variability across all models, revealing a high sensitivity to contextual noise and underscoring substantial reliability concerns for clinical deployment.
📝 Abstract
Large language models (LLMs) are increasingly utilized in clinical reasoning and risk assessment. However, their interpretive reliability in critical and indeterminate domains such as psychiatry remains unclear. Prior work has identified algorithmic biases and prompt sensitivity in these systems, raising concerns about how contextual information may influence model outputs, but there remains no systematic way to assess these, especially in the psychiatric domain. We propose an approach for reliability auditing downstream LLM tasks by structuring evaluation around the impact of prompt design and the inclusion of medically insignificant inputs on predicted hospitalization risk scores, which is often the first downstream AI clinical-decision-making task. In our audit, a cohort of synthetic patient profiles (n = 50) is generated, each consisting of 15 clinically relevant features and up to 50 clinically insignificant features, across four prompt reframings (neutral, logical, human impact, clinical judgment). We audit four LLMs (Gemini 2.5 Flash, LLaMa 3.3 70b, Claude Sonnet 4.6, GPT-4o mini), and our results show that including medically insignificant variables resulted in a statistically significant increase in the absolute mean predicted hospitalization risk and output variability across all models and prompts, indicating reduced predictive stability as contextual noise increased. Clinically insignificant features had an effect on instability across many model-prompt conditions, and prompt variations independently affected the trajectory of instability in a model-dependent manner. These findings quantify how LLM-based psychiatric risk assessments are sensitive to non-clinical information, highlighting the need for systematic evaluations of attributional stability and uncertainty behavior like this before clinical deployments.