Learning in Games with progressive hiding

2024-09-05

arXiv.org

AI Summary

In imperfect-information games, simultaneously learning game rules and respecting information constraints remains challenging. Method: This paper proposes a progressive hiding mechanism: initially revealing private information to learn action effects and game structure, then gradually introducing information masking to achieve a smooth transition from fully observable to realistic imperfect-information settings. It is the first to incorporate stochastic multi-stage optimization principles—such as scenario decomposition and progressive hedging—into game-theoretic learning frameworks, and theoretically extends Counterfactual Regret Minimization (CFR) to games with imperfect recall. Results: Evaluated on a novel communication trading game benchmark, the method significantly accelerates strategy convergence, enhances robustness, and achieves optimal expected payoff, thereby demonstrating both effectiveness and generalizability across imperfect-information domains.

Technology Category

• Game Theory and Economic Paradigms: Imperfect Information
• Search and Optimization: Learning to Search
• Multiagent Systems: Mechanism Design

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

When learning to play an imperfect information game, it is often easier to first start with the basic mechanics of the game rules. For example, one can play several example rounds with private cards revealed to all players to better understand the basic actions and their effects. Building on this intuition, this paper introduces {it progressive hiding}, an algorithm that learns to play imperfect information games by first learning the basic mechanics and then progressively adding information constraints over time. Progressive hiding is inspired by methods from stochastic multistage optimization such as scenario decomposition and progressive hedging. We prove that it enables the adaptation of counterfactual regret minimization to games where perfect recall is not satisfied. Numerical experiments illustrate that progressive hiding can achieve optimal payoff in a benchmark of emergent communication trading game.

Problem

• Balancing learning mechanics and information constraints in games
• Adapting regret minimization to non-perfect recall games
• Improving performance via progressive hiding algorithm

Innovation

• Progressive hiding balances learning and constraints
• Uses stochastic multistage optimization methods
• Adapts counterfactual regret minimization effectively