LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High

2025-05-28

AI Summary

This study investigates large language models’ (LLMs) ability to identify and reject false presuppositions in high-stakes political contexts. Addressing the lack of systematic evaluation of implicit misleading assumptions in prior work, we integrate linguistic presupposition theory with political misinformation scenarios to construct a dedicated benchmark dataset and design controlled experiments. We systematically evaluate GPT-4-o, Llama-3-8B, and Mistral-7B-v0.3. Results show that all three models struggle significantly to reject false presuppositions; accuracy is markedly influenced by syntactic structure, partisan framing, and event prior probability. Cross-model variations in presupposition sensitivity reveal a critical vulnerability to implicit manipulation. Our work establishes “presupposition analysis” as a novel paradigm for detecting the reinforcement effects of political misinformation, providing both theoretical grounding and empirical evidence for assessing and improving LLM robustness against presuppositionally embedded deception.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper examines how LLMs handle false presuppositions and whether certain linguistic factors influence their responses to falsely presupposed content. Presuppositions subtly introduce information as given, making them highly effective at embedding disputable or false information. This raises concerns about whether LLMs, like humans, may fail to detect and correct misleading assumptions introduced as false presuppositions, even when the stakes of misinformation are high. Using a systematic approach based on linguistic presupposition analysis, we investigate the conditions under which LLMs are more or less sensitive to adopt or reject false presuppositions. Focusing on political contexts, we examine how factors like linguistic construction, political party, and scenario probability impact the recognition of false presuppositions. We conduct experiments with a newly created dataset and examine three LLMs: OpenAI's GPT-4-o, Meta's LLama-3-8B, and MistralAI's Mistral-7B-v03. Our results show that the models struggle to recognize false presuppositions, with performance varying by condition. This study highlights that linguistic presupposition analysis is a valuable tool for uncovering the reinforcement of political misinformation in LLM responses.

Problem

• LLMs fail to detect false presuppositions in high-stakes misinformation
• Linguistic factors influence LLM responses to falsely presupposed content
• LLMs reinforce political misinformation due to poor presupposition recognition

Innovation

• Analyzing LLMs with linguistic presupposition techniques
• Testing models on political misinformation scenarios
• Evaluating GPT-4, Llama-3, Mistral-7B performance