Last-Iterate Convergence of No-Regret Learning for Equilibria in Bargaining Games

2025-07-03

AI Summary

This paper investigates the last-iterate convergence of no-regret learning algorithms to Nash equilibria in bargaining games. Building upon the Follow-the-Regularized-Leader (FTRL) framework, it establishes, for the first time, rigorous finite-time convergence to ε-Nash equilibria in both single-shot and repeated bargaining games—without imposing structural assumptions on the game (e.g., zero-sum or monotonicity) or requiring algorithm-specific hyperparameter tuning. The theoretical contribution lies in eliminating reliance on restricted game classes, thereby substantially broadening the applicability of no-regret learning. Explicit convergence time bounds are provided. Empirical evaluation confirms robust convergence across symmetric and asymmetric payoff structures and diverse initial conditions, demonstrating how simple learning rules spontaneously induce complex economic behaviors—including threat strategies and multiplicity of equilibria.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Mechanism Design
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Bargaining games, where agents attempt to agree on how to split utility, are an important class of games used to study economic behavior, which motivates a study of online learning algorithms in these games. In this work, we tackle when no-regret learning algorithms converge to Nash equilibria in bargaining games. Recent results have shown that online algorithms related to Follow the Regularized Leader (FTRL) converge to Nash equilibria (NE) in the last iterate in a wide variety of games, including zero-sum games. However, bargaining games do not have the properties used previously to established convergence guarantees, even in the simplest case of the ultimatum game, which features a single take-it-or-leave-it offer. Nonetheless, we establish that FTRL (without the modifications necessary for zero-sum games) achieves last-iterate convergence to an approximate NE in the ultimatum game along with a bound on convergence time under mild assumptions. Further, we provide experimental results to demonstrate that convergence to NE, including NE with asymmetric payoffs, occurs under a broad range of initial conditions, both in the ultimatum game and in bargaining games with multiple rounds. This work demonstrates how complex economic behavior (e.g. learning to use threats and the existence of many possible equilibrium outcomes) can result from using a simple learning algorithm, and that FTRL can converge to equilibria in a more diverse set of games than previously known.

Problem

• Study no-regret learning convergence in bargaining games
• Analyze FTRL's last-iterate convergence to Nash equilibria
• Explore equilibria in ultimatum and multi-round bargaining games

Innovation

• FTRL achieves last-iterate convergence
• Convergence to approximate Nash equilibria
• Works in bargaining and ultimatum games