Misaligned Clinical Risk Classification and Cost Asymmetry in Open-Weight Large Language Models

2026-09-20

AI Summary

研究探讨了四种大型语言模型在处理临床风险与成本权衡时的表现，发现这些模型虽能编码风险和成本信息，但未能可靠地将其整合为正确的成本决策。

Abstract

How large language models (LLMs) integrate patient risk with clinical cost tradeoffs remains poorly understood. We investigated how four open-weight LLMs (Qwen-2.5-7B/32B and Llama-3.1-8B/70B) internally represent cost tradeoffs, how these representations relate to clinical predictions, and whether decisions shift as predicted by the specified cost direction and magnitude. Using a public diabetes dataset, we varied 11 false-negative (FN) to false-positive (FP) cost ratios across three phrasings and examined representations and behavioral outputs. Patient risk was linearly recoverable on par with conventional classifiers (AUC $\approx 0.83$), and cost direction was recoverable in every model. However, representational shifts in cost direction tracked output changes only in the two larger models, and responses to cost magnitude were predominantly direction-agnostic. Only 2 of 12 model-phrasings showed both opposing responses to increasing FN versus FP costs and cost-correct ordering. Representationally, a direction fitted on one cost side did not invert when transferred to the other, as expected under mirror-symmetric encoding. These findings suggest that LLMs encode risk and cost information but do not reliably integrate them into cost-correct decisions. Clinical evaluations should therefore include tradeoff tests, phrasing sensitivity, and default operating points alongside predictive performance.

Problem

• Large Language Models
• Clinical Cost Tradeoffs
• Patient Risk

Innovation

• cost tradeoffs
• clinical predictions
• large language models (LLMs)
• risk representation
• decision-making