The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

2026-06-01

AI Summary

This work addresses the critical challenge of optimally allocating large language model (LLM) inference resources under strict computational budget constraints to maximize overall performance. Framing the problem as a globally constrained optimization task grounded in economic principles, the authors propose CLEAR (Constrained Latent-utility Equilibrium Allocation for Reasoning), an algorithm that models per-query inference utility via a shifted surge function and employs a shadow pricing mechanism to balance marginal utilities under resource scarcity. This approach enables rational abandonment of low-potential queries while directing precise investment toward high-potential ones. By introducing economic equilibrium concepts into LLM inference scheduling for the first time, CLEAR significantly advances the cost–accuracy Pareto frontier across diverse tasks and traffic scenarios, achieving up to a threefold improvement in global accuracy over uniform allocation under extreme resource constraints.

Abstract

Inference-time scaling has emerged as a critical avenue for enhancing Large Language Models' performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles. By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds. Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.

Problem

• inference budget allocation
• Large Language Models
• resource scarcity
• reasoning utility
• computational budgets

Innovation

• shadow price
• inference-time scaling
• budget allocation
• marginal utility
• CLEAR