🤖 AI Summary
This work addresses the decision-making challenges in Big 2, a four-player imperfect-information card game characterized by hidden information, sparse rewards, and non-stationary opponents. The authors propose a unified self-play reinforcement learning framework to systematically compare policy gradient and value approximation methods. Their findings indicate that moderate entropy regularization effectively mitigates excessive policy determinism, and that training against the current policy outperforms checkpoint-based or fixed-opponent mechanisms under limited training budgets. Empirical results demonstrate that Proximal Policy Optimization (PPO) with entropy regularization significantly outperforms alternative algorithms—including MC-Q, SARSA, and Q-learning—when facing random, greedy, and heuristic opponents. These results validate Big 2 as a compelling benchmark for multi-agent reinforcement learning in imperfect-information settings.
📝 Abstract
Imperfect-information multiplayer games test whether agents can act under hidden information, sparse rewards, and non-stationary opponents. We study these challenges in Big 2, a four-player imperfect-information card game. We develop a self-play RL framework for Big 2 that enables controlled comparisons between policy-gradient and value-approximating agents. Under a common environment, input representation, training budget, and evaluation protocol, PPO outperforms Monte Carlo Q approximation, SARSA, and Q-learning against random, greedy, and heuristic Big 2 opponents. We further find that moderate entropy regularization improves PPO by preventing the policy from becoming overly deterministic, and that current-policy self-play provides a stronger finite-budget curriculum than checkpoint self-play or fixed-opponent training. Together, these results show that Big 2 is a useful controlled setting for studying deep RL under imperfect information, multiplayer interaction, delayed rewards, and variable action sets.