Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking

2026-08-07

AI Summary

This work investigates why large reasoning models, despite solving the standard Tower of Hanoi problem effectively, struggle with its flat-to-flat variant. By training small Transformers and applying interpretability techniques—including linear probing and representation injection—the authors analyze internal representations of Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B. They find that these models construct geometrically faithful world models resembling the Sierpiński triangle but suffer from representational degradation during inference, leading to failure. Injecting the learned representation early in the reasoning process substantially improves success rates on multi-ring tasks. These findings indicate that the performance bottleneck lies not in the ability to construct an accurate world model, but in maintaining it throughout inference.

Abstract

The Tower of Hanoi is a simple planning puzzle that in prior work has proven challenging for large reasoning models (LRMs). Current models solve the standard formulation of the puzzle, but still struggle with the flat-to-flat variant (where initial and goal states are not restricted to have all rings on a single peg). This paper presents an in-depth study of how both small, in-house Transformers and large, third-party LRMs solve this task. To understand the failures mechanistically, we first train small Transformers from scratch on precomputed solution traces. Using a variety of interpretability techniques, we show that these Transformers develop an emergent world model: a linearly decodable, geometrically faithful representation of the puzzle's state space (the Sierpinski triangle), that is causally involved in solving the puzzles. Second, we return to the large LLMs and apply our techniques to two frontier reasoning models, Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B, that attempt to solve the task through extended chain-of-thought. Surprisingly, we find that both models encode the Sierpinski world model near-perfectly at the end of the prompt, and yet fail at the majority of tasks when there are more than 3 rings. We locate the source of this failure in the decaying representation of the world model. We probe for the representation at different stages during planning, and establish causality by showing that performance can be improved by injecting the prompt-time representation at inference. The failure of the models is thus one of maintenance of the required representations, not their absence, and performance is at least partially recoverable. These results thus reframe the reported collapse in performance from prior work: current Large Reasoning Models build a world model, and then lose it.

Problem

• Tower of Hanoi
• world model
• large reasoning models
• representation maintenance
• planning failure

Innovation

• emergent world model
• representation decay
• Tower of Hanoi
• mechanistic interpretability
• state space geometry