Technology Category
Cognitive Modeling & Cognitive Systems: Conceptual Inference and ReasoningNatural Language Processing: (Large) Language ModelsMachine Learning: Large Multimodal Models (LMMs)
GPT-4o Lacks Core Features of Theory of Mind
🤖 AI Summary
Whether current large language models genuinely possess theory of mind (ToM)—the capacity to coherently, generically, and consistently model the causal relationships between mental states and behavior—remains contentious. This work proposes a cognitively grounded evaluation paradigm rooted in cognitive science definitions of ToM, moving beyond surface-level task performance to examine the internal causal structure of model representations. Through methods including logical-equivalence task comparisons and consistency analyses between behavioral predictions and mental-state inferences, the study reveals that while models such as GPT-4o can mimic human judgments on simple ToM tasks, they exhibit inconsistent performance under logically equivalent conditions and display a marked dissociation between behavioral and mental-state reasoning. These findings indicate a fundamental absence of core ToM capabilities in existing models.
Technology Category
Application Category
📝 Abstract
Do Large Language Models (LLMs) possess a Theory of Mind (ToM)? Research into this question has focused on evaluating LLMs against benchmarks and found success across a range of social tasks. However, these evaluations do not test for the actual representations posited by ToM: namely, a causal model of mental states and behavior. Here, we use a cognitively-grounded definition of ToM to develop and test a new evaluation framework. Specifically, our approach probes whether LLMs have a coherent, domain-general, and consistent model of how mental states cause behavior -- regardless of whether that model matches a human-like ToM. We find that even though LLMs succeed in approximating human judgments in a simple ToM paradigm, they fail at a logically equivalent task and exhibit low consistency between their action predictions and corresponding mental state inferences. As such, these findings suggest that the social proficiency exhibited by LLMs is not the result of an domain-general or consistent ToM.
Problem
Research questions and friction points this paper is trying to address.
Theory of Mind
Large Language Models
mental states
causal model
behavior prediction
Innovation
Methods, ideas, or system contributions that make the work stand out.
Theory of Mind
Large Language Models
causal model
cognitive evaluation
mental state inference
🔎 Similar Papers
No similar papers found.
