Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learning

2026-08-03

AI Summary

This work addresses the challenge of modeling curling tactical decision-making under continuous state and action spaces, stochastic action outcomes, and high sensitivity to state transitions. We propose the first self-supervised deep reinforcement learning framework for this domain, integrating finite-horizon backward induction into an enhanced Deep Deterministic Policy Gradient (DDPG) algorithm to learn and evaluate strategies without human-annotated data. The trained agent achieves performance comparable to expert heuristic policies in a four-stone simplified curling environment. Moreover, the critic network provides dense value estimates across the entire action space, enabling post-game analysis and training support. This approach facilitates, for the first time, quantitative evaluation of strategic factors such as the advantage of playing first.

Abstract

Curling is often referred to as "Chess on Ice", owing to the tactical complexity of its decision-making process. Yet unlike chess, curling remains largely underexplored from a machine learning perspective, with prior work confined mainly to statistical approaches. We propose a reinforcement learning framework capable of quantitatively evaluating and comparing tactical options in curling. The game poses several modeling challenges: continuous state and action spaces, stochastic action outcomes reflecting player skill variability, and state transitions that are highly sensitive to small perturbations in the executed action. To address them, we employ the Deep Deterministic Policy Gradient actor-critic algorithm, adapted to exploit the finite-horizon structure of the game. Our experiments show that effective curling strategies can be acquired in a fully self-supervised manner, without any human-annotated data: on a reduced four-rock variant, the learned agent matches a hand-crafted expert heuristic in a regime where that heuristic is close to optimal, a parity we quantify against the intrinsic hammer advantage of the variant. Beyond the resulting policy, the learned critic provides a dense value estimate over the entire continuous action space, enabling the quantitative comparison of tactical alternatives for applications such as post-game performance analysis and decision support during athlete preparation.

Problem

• curling
• tactical decision-making
• continuous action space
• stochastic outcomes
• reinforcement learning

Innovation

• Deep Reinforcement Learning
• Curling Strategy
• Continuous Action Space
• Self-Supervised Learning
• Tactical Decision-Making