From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning

2026-06-08

AI Summary

This work addresses the susceptibility of existing theory-of-mind (ToM) post-training methods to spurious correlations—often termed “shortcuts”—which undermine models’ genuine reasoning capabilities. To tackle this issue, the authors propose Thinking-RFT, a novel framework that systematically identifies and circumvents such shortcuts in ToM tasks. By integrating reinforcement fine-tuning (RFT) with explicit reasoning chains, a verifiable reward mechanism, and anchor-based cue guidance, Thinking-RFT enables robust post-training on shortcut-free ToM datasets. Experimental results demonstrate that, compared to supervised fine-tuning, Thinking-RFT yields a 6% overall improvement on ToM tasks, with gains of 10% in higher-order belief and intention reasoning and 7% in multimodal scenarios. Moreover, the approach exhibits superior generalization and robustness in unseen domains and counterfactual settings.

Abstract

Theory of Mind (ToM) is a must-acquire skill for modern foundation model systems to operate effectively and safely in the real world. Recent works have explored honing ToM via post-training; however, we show that such progress is confounded by a pervasive "shortcut" issue: tasks can reach up to 99% accuracy by simply exploiting spurious causal correlations, leading to a false sense of ToM. Motivated by this, we first develop a framework to systematically examine ToM datasets for shortcuts and provide guidance for future development. We find that questions reducible to pure state tracking, such as "belief," are especially shortcut-prone compared to mind questions, such as "intention," where reasoning beyond tracking is required. Using four shortcut-free datasets across three ToM contexts, we then comprehensively study whether Reinforcement Fine-Tuning with verifiable rewards and explicit reasoning chains, called Thinking-RFT, elevates ToM beyond Supervised Fine-Tuning, or SFT. Our key findings are as follows. First, Thinking-RFT effectively improves ToM in all scenarios, with a 6% improvement over SFT, particularly in complex higher-order reasoning, with a 10% improvement over SFT, and multimodal cases, with a 7% improvement over SFT. It also generalizes notably better to unseen domains and higher-order queries while being more robust to counterfactuals. Second, ToM benefits specifically from the joint effect of reasoning and RL: Thinking-RFT outperforms Non-Thinking-RFT by 7% on average. Third, RFT works by learning to ground its reasoning on anchor cues, such as keywords and state changes, that correspond to causal factors. We believe our study is useful for developing effective and robust ToM post-training datasets and advancing critical ToM capabilities.

Problem

• Theory of Mind
• shortcut learning
• spurious correlations
• reasoning
• post-training

Innovation

• Theory of Mind
• Reinforcement Fine-Tuning
• Reasoning Chains
• Shortcut Mitigation
• Robust Post-Training