The Good, the Bad, and the Ugly: The Role of AI Quality Disclosure in Lie Detection

2026-04-11

AI Summary

This study investigates how opaque AI assistant quality exacerbates cognitive biases in middle school students’ detection of misinformation. Method: A behavioral experiment simulating deceptive social media dialogues was conducted, employing a human-AI collaborative evaluation framework to quantify lie-detection accuracy and expectation bias. Contribution/Results: We provide the first empirical evidence that, in the absence of performance disclosure, low-quality AI significantly reduces lie-detection accuracy by 17.3% below individual baselines; transparent disclosure of actual performance fully restores trust equilibrium and judgment accuracy to baseline levels. In contrast, high-quality AI consistently enhances detection performance irrespective of disclosure. Critically, we establish AI quality transparency as a key causal variable for information literacy interventions, offering rigorous causal evidence and actionable design principles for fostering trustworthy human-AI collaboration in educational settings.

Technology Category

Natural Language Processing: Ethics — Bias, Fairness, Transparency & PrivacyPhilosophy and Ethics of AI: Safety, Robustness & TrustworthinessHumans and AI: Learning Human Values and Preferences

Application Category

Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAIResponsible Web: Algorithmic accountability and transparency on the webUser Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

We investigate how low-quality AI advisors, lacking quality disclosures, can help spread text-based lies while seeming to help people detect lies. Participants in our experiment discern truth from lies by evaluating transcripts from a game show that mimicked deceptive social media exchanges on topics with objective truths. We find that when relying on low-quality advisors without disclosures, participants' truth-detection rates fall below their own abilities, which recovered once the AI's true effectiveness was revealed. Conversely, high-quality advisor enhances truth detection, regardless of disclosure. We discover that participants' expectations about AI capabilities contribute to their undue reliance on opaque, low-quality advisors.

Problem

• Cognitive Bias
• AI Assistant Reliability
• Quality Disclosure

Innovation

• AI Transparency
• Quality of AI Assistants
• User Trust and Credibility Assessment