Mental-R1: Aligning LLM Reasoning for Mental Health Assessment

2026-06-11

AI Summary

Current large language models lack reasoning mechanisms aligned with human cognitive processes in mental health assessment, leading to unreliable and uninterpretable outcomes. This work proposes the Cognitive-relative Policy Optimization (CRPO) framework, which integrates cognitive appraisal theory into reinforcement learning for the first time. By modeling stage-dependent uncertainty and applying staged entropy regularization, CRPO formally simulates the human cognitive reasoning trajectory from uncertainty to certainty. Evaluated across eight mental health datasets, the approach achieves an average improvement of 10.4 percentage points in weighted F1 score over the strongest reinforcement learning baseline. The resulting Mental-R1 model significantly outperforms existing large language models in complex reasoning scenarios, demonstrating enhanced alignment with human-like reasoning and improved interpretability.

Abstract

Mental health problems such as anxiety, depression, and suicide remain urgent global challenges, where timely and accurate assessment is critical for effective intervention. Recently, large language models have been explored for mental health assessment. However, existing general-purpose post-training methods do not align with the cognitive processes of human assessment, which may lead to unreliable reasoning outcomes. To bridge this gap, we propose Cognitive Relative Policy Optimization (CRPO), a reinforcement learning framework tailored for the mental health domain. CRPO extends group relative policy optimization by integrating stage-dependent uncertainty modeling into the policy optimization process. Specifically, we introduce a stage-wise entropy regularization mechanism that encourages broad exploration in early reasoning phases and progressively enforces confident decision-making in later stages, mimicking the human cognitive shift from uncertainty to certainty. In addition, inspired by cognitive appraisal theory, we formalize cognitive reasoning stages, thereby guiding theory-grounded interpretable inference. Experiments on 8 mental health datasets show that CRPO achieves an average improvement of 10.4 percentage points in weighted F1-score over the best reinforcement learning baseline. Furthermore, the CRPO-trained model Mental-R1 demonstrates clear advantages compared with existing large language models on reasoning-intensive cases, suggesting that CRPO enhances reasoning capabilities for mental health assessment.

Problem

• mental health assessment
• large language models
• cognitive alignment
• reasoning reliability
• human cognitive processes

Innovation

• Cognitive Relative Policy Optimization
• stage-wise entropy regularization
• cognitive appraisal theory
• reasoning alignment
• mental health assessment