LeAct: Learning to Reason from Expert Actions

2026-07-23

AI Summary

This work addresses the challenge of distilling generalizable natural language reasoning capabilities from “silent” expert systems that provide only optimal actions without revealing their underlying reasoning processes. Treating the chain-of-thought behind expert decisions as a latent variable, the proposed method employs a student model to sample candidate reasoning trajectories and retains those that significantly increase the likelihood of reproducing the expert’s actions, thereby inverting action selection into interpretable reasoning. This approach establishes the first general framework for transforming non-verbal experts into teachers capable of conveying structured reasoning. Empirical results demonstrate that the method approaches the performance lower bound of exact solvers in small enumerable games, achieves a +60 mbb/g advantage in Flop Hold’em (a game with approximately 10⁹ information sets), and substantially outperforms baseline approaches such as direct imitation in robotic tasks.

Abstract

Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs. However, a rich and largely untapped source of supervision lies in expert systems (e.g., game engines, classical planners, theorem provers), which routinely produce near-optimal actions across diverse domains. But these experts are silent: they commit to an action without writing down the chain of thought (CoT) behind it. Recovering that CoT as natural-language reasoning would distill expert knowledge into a student that generalizes beyond the demonstrated actions. We treat it as a latent variable and study how to recover it from the action alone. Our approach, LeAct (Learning to reason from Actions), optimizes this latent variable: the student samples candidate CoTs for each expert action, and we retain those that measurably improve its own probability of recovering the action. Across imperfect-information games at multiple scales and a simulated robotics benchmark, LeAct reaches the solver's numerical floor on small enumerable games. At larger scale, it is $5\times$ closer to the solver than the strongest expert-iteration baseline. At Flop Hold'em ($\sim 10^9$ infosets), LeAct wins head-to-head by $+60$ mbb/g, and on the robotics probe it is the only training recipe that improves on direct imitation. We present a principled framework and the result: expert systems become a categorically new source of reasoning teachers for foundation models.

Problem

• expert systems
• chain-of-thought
• reasoning
• action supervision
• knowledge distillation

Innovation

• Chain-of-Thought
• Expert Systems
• Latent Reasoning
• Imitation Learning
• Reasoning Distillation