Measuring Choice Difficulty

2026-04-29

AI Summary

This study investigates the intrinsic relationships and cross-domain transferability among prevalent measures of choice difficulty—such as comprehensibility, response randomness, and post-decision confidence—within a binary-choice Bayesian expected utility framework. Integrating Blackwell’s experiment model with psychophysical task modeling, the work demonstrates that these widely used difficulty metrics are generally uncorrelated, and identifies intuitive sufficient conditions under which their ordinal rankings align. A key contribution is the proof that, in psychophysical tasks rewarding only correctness, confidence and comprehensibility are equivalent. Furthermore, the study establishes that willingness to pay—measured in utility units—for switching options precisely quantifies comprehensibility. By delineating the theoretical boundaries of each difficulty measure, this research provides a rigorous foundation for interdisciplinary experimental design and interpretation, while cautioning against their misuse or unwarranted generalization.

Abstract

We provide a theoretical framework to understand how widely used measures of choice difficulty relate. In a binary-option Bayesian expected-utility framework, we show that three measures of difficulty, (i) understanding (ex-ante value), (ii) choice randomness, and (iii) confidence that the chosen option is ex post correct, are, in general, unrelated, and that this result extends to other potential measures like attenuation. We provide intuitive sufficient conditions which align the orders, using both restrictions on Blackwell experiments that capture well known classes (such as logit) and restrictions on payoffs and demonstrate that in psychophysical tasks that pay only for correctness, confidence coincides with understanding. We show willingness-to-accept to switch, when measured in utils, is equivalent to understanding. Our results suggest caution in interpreting measures of choice difficulty as well as the degree of portability between economics and psychophysics experiments

Problem

• choice difficulty
• Bayesian decision theory
• confidence
• understanding
• randomness

Innovation

• choice difficulty
• Bayesian expected utility
• Blackwell experiments
• psychophysics
• decision confidence