Search-contempt: a hybrid MCTS algorithm for training AlphaZero-like engines with better computational efficiency

2025-04-10

AI Summary

This work addresses the prohibitively high computational cost of self-play training in AlphaZero-style engines. We propose *search-contempt*, a novel hybrid Monte Carlo Tree Search (MCTS) algorithm integrating a dynamic “contempt” mechanism into node selection. Its core innovation is the first incorporation of adaptive contempt bias—guided by position difficulty estimation and a modified PUCT heuristic—into MCTS to actively steer self-play toward more challenging and diverse positions. This yields higher-quality training data and significantly improves generalization, especially in asymmetric games such as Odds Chess. Empirical evaluation demonstrates that search-contempt achieves strong engine performance using only hundreds of thousands of self-play games—reducing the sample requirement by two orders of magnitude compared to AlphaZero—enabling full training on consumer-grade GPUs at a total cost under $10,000, while outperforming baseline models.

Technology Category

• Search and Optimization: Adversarial Search
• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

AlphaZero in 2017 was able to master chess and other games without human knowledge by playing millions of games against itself (self-play), with a computation budget running in the tens of millions of dollars. It used a variant of the Monte Carlo Tree Search (MCTS) algorithm, known as PUCT. This paper introduces search-contempt, a novel hybrid variant of the MCTS algorithm that fundamentally alters the distribution of positions generated in self-play, preferring more challenging positions. In addition, search-contempt has been shown to give a big boost in strength for engines in Odds Chess (where one side receives an unfavorable position from the start). More significantly, it opens up the possibility of training a self-play based engine, in a much more computationally efficient manner with the number of training games running into hundreds of thousands, costing tens of thousands of dollars (instead of tens of millions of training games costing millions of dollars required by AlphaZero). This means that it may finally be possible to train such a program from zero on a standard consumer GPU even with a very limited compute, cost, or time budget.

Problem

• Improves computational efficiency in AlphaZero-like self-play training
• Enhances strength in Odds Chess with challenging positions
• Enables training on consumer GPUs with limited budgets

Innovation

• Hybrid MCTS algorithm for efficient self-play
• Prefers challenging positions in training
• Reduces training cost to thousands of dollars