OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data

2025-10-16

AI Summary

Real-world knowledge work (e.g., healthcare, finance) demands probabilistic reasoning and numerical estimation under incomplete information and uncertainty, yet existing benchmarks predominantly evaluate deterministic tasks and lack systematic evaluation of language models’ ability to represent and calibrate prior distributions. Method: We introduce OpenEstimate—the first scalable, multi-domain benchmark for realistic uncertain reasoning—built on real-world distributional data, quantifying models’ probabilistic prior expression along both accuracy and calibration dimensions. Our evaluation employs multi-turn prompting, controllable reasoning depth, and diverse sampling strategies. Results: Experiments across six state-of-the-art large language models reveal pervasive prior miscalibration and significant overconfidence; existing uncertainty-guided techniques yield only marginal improvements. This work establishes a new evaluation paradigm and standardized benchmark for uncertainty-aware reasoning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Real-world settings where language models (LMs) are deployed -- in domains spanning healthcare, finance, and other forms of knowledge work -- require models to grapple with incomplete information and reason under uncertainty. Yet most LM evaluations focus on problems with well-defined answers and success criteria. This gap exists in part because natural problems involving uncertainty are difficult to construct: given that LMs have access to most of the same knowledge as humans, it is non-trivial to design questions for which LMs will struggle to produce correct answers, but which humans can answer reliably. As a result, LM performance on reasoning under uncertainty remains poorly characterized. To address this gap, we introduce OpenEstimate, an extensible, multi-domain benchmark for evaluating LMs on numerical estimation tasks that require models to synthesize significant amounts of background information and express predictions as probabilistic priors. We assess these priors for accuracy and calibration, quantifying their usefulness relative to samples from the true distribution of interest. Across six frontier LMs, we find that LM-elicited priors are often inaccurate and overconfident. Performance improves modestly depending on how uncertainty is elicited from the model, but is largely unaffected by changes in sampling strategy, reasoning effort, or prompt design. The OpenEstimate benchmark thus offers a challenging evaluation for frontier LMs and a platform for developing models that are better at probabilistic estimation and reasoning under uncertainty.

Problem

• Evaluating LLMs on reasoning under uncertainty with real-world data
• Assessing model performance on numerical estimation with probabilistic priors
• Addressing the gap in LM evaluations for incomplete information scenarios

Innovation

• Introduces OpenEstimate benchmark for uncertainty evaluation
• Assesses LM priors for accuracy and calibration
• Evaluates probabilistic estimation across multiple domains