Lost in Delusion: Examining LLM Safety Under User Delusions and Distress

2026-05-30

AI Summary

Current large language models lack reliable safety mechanisms when users simultaneously express psychological distress and delusional beliefs, with intervention efficacy markedly declining in multi-turn dialogues. This study constructs clinically plausible multi-turn conversation simulations, employing matched-control designs, clinical role framing, a delusion classifier, and tailored prompt engineering to systematically evaluate behavioral differences across six mainstream models in scenarios involving delusions versus distress alone. It reveals, for the first time, a critical “recognition–intervention” disconnect: the presence of delusional content reduces safety interventions by up to 4.5-fold. The work proposes delusion-aware prompting and explicit response guidance strategies that partially mitigate this gap, though their effectiveness remains constrained by the reliability of the delusion classifier on vulnerable models.

Abstract

LLM chatbots increasingly serve as a first source of support for people in psychological distress, including those whose distress is entangled with delusional beliefs. Prior work on LLM mental-health safety largely evaluates general therapeutic quality or single-turn crisis detection, leaving unclear how models behave when distress is intertwined with delusion over sustained conversations. We address this gap with matched multi-turn simulations, across clinically grounded personas and six LLMs, that pair each delusional conversation with a distress-only control to isolate the effect of delusional framing. This reveals a recognition-intervention gap: models detect distress at comparable rates regardless of framing, yet sharply fail to act on it once distress is embedded in delusion, with safety interventions suppressed by up to 4.5x. The failure tracks accumulated acceptance of the user's premises rather than emotional validation. Worse, the intuitive fix of prompting models to assess user distress backfires under delusional framing; only delusion-aware prompting with explicit response guidance closes the gap, and even this depends on a delusion classifier that is itself unreliable on the most vulnerable models. Safe deployment therefore requires treating delusional framing as a distinct risk signal that overrides conversational accommodation.

Problem

• delusion
• psychological distress
• LLM safety
• multi-turn conversation
• safety intervention

Innovation

• delusion-aware prompting
• recognition-intervention gap
• multi-turn simulation
• LLM safety
• delusional framing