Diagnosing and Improving Probabilistic Reasoning in Large Language Models

2026-09-29

AI Summary

This study addresses the deficiency of large language models in probabilistic reasoning for decision support. We introduce a novel decision-theoretic loss decomposition framework that partitions decision loss into belief formation and action optimization components. By integrating synthetic benchmarks with reinforcement learning interventions, we conduct systematic evaluations across frontier and open-source models. Our findings reveal that single-objective optimization merely redistributes losses rather than improving overall performance, whereas joint optimization requires alignment between training and evaluation formats. Furthermore, this work elucidates the differential effects and transferability of reinforcement learning on beliefs versus actions, providing critical theoretical foundations and practical guidance for comprehensively enhancing probabilistic reasoning in large language models.

Abstract

Large language models (LLMs) are increasingly proposed as decision assistants who must reason probabilistically from available evidence under explicit decision costs. We propose a decision-theoretic framework that decomposes LLMs' decision loss into two components: forming accurate beliefs from provided evidence and translating those beliefs into actions that optimize a provided utility function. Using a synthetic benchmark with known ground truth, we apply the decomposition to characterize probabilistic reasoning in frontier and open-sourced models. We further evaluate whether RL interventions targeting beliefs, decisions, or both improve these components across three domains, whether improvements transfer across components and elicitation formats, and whether decision performance can improve without improvement in belief formation. We find that targeting one component of probabilistic reasoning redistributes decision loss, improving the target without necessarily transferring to others, and that jointly targeting belief formation and decision-making improves both but hinges on matched formats between training and evaluation.

Problem

• Probabilistic Reasoning
• Large Language Models
• Decision Loss
• Belief Formation
• Decision-making

Innovation

• Probabilistic Reasoning
• Decision-Theoretic Framework
• Reinforcement Learning
• Belief-Action Decomposition
• Large Language Models