Teaching an Old Dynamics New Tricks: Regularization-free Last-iterate Convergence in Zero-sum Games via BNN Dynamics

2026-02-09

AI Summary

This work addresses the limitations of existing zero-sum game solvers, which rely on regularization to ensure convergence and require meticulous hyperparameter tuning, often underperforming when payoff structures are unknown or dynamically evolving. The paper presents the first extension of Brown–von Neumann–Nash (BNN) dynamics to both normal-form and extensive-form games under stochastic noise, integrating counterfactual weighting with neural function approximation to construct a regularization-free multi-agent learning framework. The proposed method offers theoretical guarantees of last-iterate convergence, eliminates the need for hyperparameter tuning in practice, and demonstrates significant performance improvements over state-of-the-art regularization-based approaches in non-stationary environments.

Technology Category

Game Theory and Economic Paradigms: Adversarial LearningMultiagent Systems: Adversarial AgentsReasoning under Uncertainty: Stochastic Optimization

Application Category

Economics, Online Markets and Human Computation: Social networks and social learningGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Zero-sum games are a fundamental setting for adversarial training and decision-making in multi-agent learning (MAL). Existing methods often ensure convergence to (approximate) Nash equilibria by introducing a form of regularization. Yet, regularization requires additional hyperparameters, which must be carefully tuned--a challenging task when the payoff structure is known, and considerably harder when the structure is unknown or subject to change. Motivated by this problem, we repurpose a classical model in evolutionary game theory, i.e., the Brown-von Neumann-Nash (BNN) dynamics, by leveraging the intrinsic convergence of this dynamics in zero-sum games without regularization, and provide last-iterate convergence guarantees in noisy normal-form games (NFGs). Importantly, to make this approach more applicable, we develop a novel framework with theoretical guarantees that integrates the BNN dynamics in extensive-form games (EFGs) through counterfactual weighting. Furthermore, we implement an algorithm that instantiates our framework with neural function approximation, enabling scalable learning in both NFGs and EFGs. Empirical results show that our method quickly adapts to nonstationarities, outperforming the state-of-the-art regularization-based approach.

Problem

• zero-sum games
• regularization
• Nash equilibrium
• hyperparameter tuning
• nonstationarity

Innovation

• BNN dynamics
• last-iterate convergence
• regularization-free
• extensive-form games
• counterfactual weighting