Three-Body Alignment: Aligning Chess Agent with Human Reasoning through Reranked Rationale

2026-07-24

AI Summary

This work proposes a “Tripartite Alignment” framework to enhance the alignment between complex reasoning agents and human conceptual models, leveraging a chess-based reasoning corpus comprising annotations from human grandmasters, engine-assisted commentators, and large language models. Through an agent-centric data engineering pipeline, t-SNE semantic embedding visualizations, and a dense retrieval–augmented narrative structure, the study reveals significant heterogeneity among the three agent types in semantic space and introduces a reranking mechanism to improve human alignment. Experimental results demonstrate that the approach effectively enhances the interpretability and human consistency of reasoning processes while preserving tactical performance. The authors also release the first structured chess reasoning dataset to support further research in this direction.

Abstract

As reasoning agents become increasingly complex, aligning their underlying reasoning and decision-making processes with human conceptual models is a challenge for AI security and safety. When modelling expert knowledge, understanding how to characterise and integrate insights from agents with fundamentally different reasoning architectures is necessary for safe and predictable deployment. We investigate this alignment through a \emph{three-body alignment} in chess, analysing the semantic divergence between rationales produced by human experts (Grandmasters), engine-assisted human commentators (who rationalise the outputs of efficiently updatable neural networks, or NNUEs), and Large Language Models (LLMs). Our contributions include: (1) A novel multisource rationale dataset, constructed using an agentic data engineering pipeline to transform unstructured expert commentary into structured, queryable data for alignment evaluation. (2) An empirical analysis of the semantic embedding space. Using t-SNE visualisation, we demonstrate that these sources form distinct clusters, confirming significant heterogeneity and reflecting fundamentally different conceptual approaches to the same environment. (3) An experiment demonstrating that reranking mechanisms can improve human alignment, while quantifying the explicit trade-off with tactical performance, offering a pathway for more interpretable agent decision-making. (4) The preliminary development of an enriched chess narrative dataset structure, designed to lay the groundwork for future evaluations of text rationale similarity and to address the limitations of standard dense retrieval. (5) Finally, we open-source our chess rationales dataset\footnote{Hugging Face: https://huggingface.co/datasets/jaymarichua/trichess} to support developing novel techniques that integrate diverse expert knowledge into human-aligned intelligent agents.

Problem

• AI alignment
• human reasoning
• chess agents
• semantic divergence
• intelligent agents

Innovation

• three-body alignment
• reranked rationale
• semantic embedding
• human-AI alignment
• multisource rationale dataset