Do Not Discretize, Optimize: Almost Greedy Fictitious Play

2026-06-10

AI Summary

This work addresses the slow convergence and reliance on strategy space discretization inherent in classical fictitious play algorithms for zero-sum games by proposing Almost Greedy Fictitious Play. The method operates directly in continuous strategy spaces, eschewing discretization, and performs an approximately greedy line search along the segment between the current empirical mixed strategy and its best response to enable more natural strategy updates. Built upon the fictitious play framework and integrating tools from continuous optimization and duality gap analysis, the algorithm achieves instance-dependent theoretical convergence guarantees. It attains a convergence rate of O(1/T) in terms of the duality gap, matching that of continuous fictitious play. Empirical results demonstrate the algorithm’s effectiveness and superiority over existing approaches.

Abstract

Our work revolves around Fictitious Play, one of the first iterative methods that is known to converge to a Nash equilibrium in zero-sum games. In recent years, there has been a revived interest, due to applications in various machine learning problems, which has motivated a line of work on its convergence properties and on proposing new variants of the initial algorithm. Our paper is along this direction and introduces one new variant, which we refer to as Almost Greedy Fictitious Play. The proposed algorithm greedily attempts to find the optimal stepsize at each iteration but its search space is constrained and includes almost all the line between the cumulative mixed strategy and the current best response. Our main result is that the method achieves an instance dependent convergence rate of $\mathcal{O}(1/T)$ with respect to the duality gap. This matches the rate of Continuous Fictitious Play, and offers an alternative to discretization. We complement our theoretical findings with experiments that demonstrate the effectiveness of the method.

Problem

• Fictitious Play
• Nash equilibrium
• zero-sum games
• convergence rate
• duality gap

Innovation

• Fictitious Play
• Almost Greedy
• Optimal Stepsize
• Convergence Rate
• Duality Gap