TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?

2025-09-21

AI Summary

This study investigates the Theory of Mind (ToM) capabilities required by large language models (LLMs) to understand white lies—socially motivated, prosocial deceptions—in complex interpersonal contexts, with emphasis on inferring altruistic intent and preserving relational harmony. To this end, we introduce TactfulToM, the first dedicated English benchmark for tactful deception understanding, constructed via a human-in-the-loop, multi-stage generative process: human-authored seed narratives are expanded by LLMs and iteratively refined by annotators to ensure authentic information asymmetry and affective tension in dialogues. Empirical evaluation reveals that state-of-the-art LLMs underperform substantially relative to human annotators, exposing fundamental deficits in deep social cognition and emotion-grounded ToM reasoning. This work constitutes the first systematic assessment of LLMs’ sociosemantic comprehension of white lies, establishing a novel paradigm and a rigorously validated benchmark for evaluating social intelligence in foundation models.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities that require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a novel English benchmark designed to evaluate LLMs' ability to understand white lies within real-life conversations and reason about prosocial motivations behind them, particularly when they are used to spare others' feelings and maintain social harmony. Our benchmark is generated through a multi-stage human-in-the-loop pipeline where LLMs expand manually designed seed stories into conversations to maintain the information asymmetry between participants necessary for authentic white lies. We show that TactfulToM is challenging for state-of-the-art models, which perform substantially below humans, revealing shortcomings in their ability to fully comprehend the ToM reasoning that enables true understanding of white lies.

Problem

• Evaluating LLMs' ability to understand white lies in conversations
• Assessing reasoning about prosocial motivations behind white lies
• Testing Theory of Mind abilities requiring nuanced social context

Innovation

• Created benchmark to evaluate LLM understanding of white lies
• Used human-in-the-loop pipeline to generate authentic conversations
• Designed conversations with information asymmetry for white lies