Explainable Fake News Detection with Large Language Model via Defense Among Competing Wisdom

2024-05-06

The Web Conference

AI Summary

Existing fake news detection methods are predominantly black-box models with limited interpretability, while existing explanation generation approaches suffer from high latency and reliance on unverified collective opinions. To address these limitations, this paper proposes the “Adversarial Collective Intelligence Defense” (ACID) framework: it decouples crowd-sourced judgments into opposing pro- and con-positions, employs large language models to generate bidirectional, verifiable attribution rationales, and introduces a rationale-based dialectical reasoning module to jointly perform detection and explanation. We pioneer the “Competitive Collective Intelligence Defense” paradigm—challenging the conventional assumption of direct aggregation of crowd opinions—and introduce, for the first time, bidirectional attribution generation coupled with dialectical inference. Evaluated on two real-world benchmarks, ACID achieves a 3.2% improvement in detection accuracy while generating natural language explanations with significantly higher consistency and faithfulness.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Adversarial Attacks & Robustness
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Most fake news detection methods learn latent feature representations based on neural networks, which makes them black boxes to classify a piece of news without giving any justification. Existing explainable systems generate veracity justifications from investigative journalism, which suffer from debunking delayed and low efficiency. Recent studies simply assume that the justification is equivalent to the majority opinions expressed in the wisdom of crowds. However, the opinions typically contain some inaccurate or biased information since the wisdom of crowds is uncensored. To detect fake news from a sea of diverse, crowded and even competing narratives, in this paper, we propose a novel defense-based explainable fake news detection framework. Specifically, we first propose an evidence extraction module to split the wisdom of crowds into two competing parties and respectively detect salient evidences. To gain concise insights from evidences, we then design a prompt-based module that utilizes a large language model to generate justifications by inferring reasons towards two possible veracities. Finally, we propose a defense-based inference module to determine veracity via modeling the defense among these justifications. Extensive experiments conducted on two real-world benchmarks demonstrate that our proposed method outperforms state-of-the-art baselines in terms of fake news detection and provides high-quality justifications.

Problem

• Explainable fake news detection
• Large language model utilization
• Defense-based inference framework

Innovation

• Evidence extraction from competing narratives
• Prompt-based justification with language model
• Defense-based inference for veracity determination