Projected Exploitability Descent for Nash Equilibrium Computation in Multiplayer Imperfect-Information Games

2026-06-27

AI Summary

This work addresses the challenge of balancing scalability and convergence in solving Nash equilibria for multiplayer imperfect-information games. The authors propose the Projected Exploitability Descent (PED) algorithm, which, for the first time, expresses the generalized exploitability function as a sum of maxima of linear functions and optimizes it via projected subgradient descent over the sequence-form strategy space, ensuring stable convergence. They further introduce FP-PED, a hybrid approach that combines fictitious play (FP) for warm-start initialization with PED for refined optimization. Experiments demonstrate that PED exhibits nearly monotonic improvement on a three-player variant of Kuhn poker, while FP-PED substantially enhances long-term convergence performance, overcoming the scalability limitations of exact methods when the deck size exceeds four.

Abstract

Many important games have more than two players and imperfect information. Existing approaches for computing Nash equilibrium, the central game-theoretic solution concept, in such games either lack scalability or obtain poor performance. In this paper we introduce a new algorithm called projected exploitability descent (PED) for approximating Nash equilibria in multiplayer games of imperfect information. The algorithm works by running projected subgradient descent minimizing a proxy for the multiplayer generalized exploitability function. The objective is nonconvex and nonsmooth, but can be represented as the sum of the maxima of linear functions, for which a subgradient can easily be computed and projected to the polytope of feasible sequence-form strategies. We explore performance of PED on a generalized version of the well-studied benchmark game three-player Kuhn poker. No prior exact algorithms scale to the version of the game with deck size larger than 4, and we compare performance to the popular algorithms of fictitious play (FP) and counterfactual regret minimization (CFR). We find that PED obtains a consistent near-monotonic improvement throughout all runs, though both FP and CFR perform significantly better in the initial iterations. This inspires a hybrid algorithm FP-PED that runs FP for an initial burn-in period before switching to PED for stable long-run refinement. We can alternatively view this as a multi-step algorithm that runs FP as a pre-processing step to obtain a strong initialization for PED.

Problem

• multiplayer games
• imperfect-information games
• Nash equilibrium
• scalability
• computational performance

Innovation

• Projected Exploitability Descent
• Nash Equilibrium
• Multiplayer Imperfect-Information Games
• Subgradient Descent
• Sequence-Form Strategies