Solver-Guided Reasoning for Mixed-Equilibrium Strategies

2026-08-06

AI Summary

This work addresses the suboptimal performance of large language models (LLMs) in game-theoretic equilibrium reasoning, which stems from their reliance on human behavioral data that often deviates from mixed-strategy equilibria. To overcome this limitation, the authors propose replacing human data with equilibrium strategies generated by game solvers to construct an interpretable Mixed-strategy Decision Tree (MDT). This framework translates implicit optimal strategies into sparse, human- and model-interpretable rules. The MDT is trained using shadow-based contrastive learning and extensive state querying—encompassing 250 million decisions. Evaluated across eight LLMs, the approach reduces the ℓ₁ distance between model policies and theoretical equilibria by 52.6% on average, demonstrating high policy fidelity and strong generalization across states in both river-end poker scenarios and Liar’s Dice tasks.

Abstract

Reasoning in large language models (LLMs) is often grounded in human text, human demonstrations, and human-generated rationales. For equilibrium reasoning in complex games, however, relying on human data can be suboptimal. In fact, human play is often guided by intuition and heuristics and can deviate substantially from game equilibrium. This discrepancy is amplified in games with mixed-strategy equilibria, where human data is heavily biased toward pure strategies. Consequently, conditioning LLMs on this data yields weak game strategies. To grant LLMs the reasoning capacity in games, in this work, we study how to elicit equilibrium play using solver output. We propose Mixed-Strategy Decision Tree (MDT), which articulates the silent optimality of the equilibrium into sparse strategic rules that both humans and LLMs could understand. Using solver output rather than human annotation allows us to extend the input to arbitrarily new states and continuations. We instantiate this study on No-Limit Texas Hold'em by querying a solver oracle for over \textbf{250 million mixed-strategy decisions}; MDT together with other techniques \textbf{reduces the $\ell_1$ distance to the equilibrium by $52.6\%$} across $8$ different LLM configurations. A Route-only ablation tests the incremental contribution of the shadow-based contrast, while complete River-endgame and Liar's Dice experiments evaluate strategic fidelity and portability beyond the original NLH communication setting.

Problem

• equilibrium reasoning
• mixed-strategy equilibrium
• large language models
• game theory
• human bias

Innovation

• Mixed-Strategy Decision Tree
• Solver-Guided Reasoning
• Equilibrium Play
• Large Language Models
• Game Theory