Exploring Modality Disruption in Multimodal Fake News Detection

2025-04-12

AI Summary

Multimodal fake news detection on social media suffers from “modality disruption”: exaggerated or stylistically embellished content in textual and visual modalities misleads discriminative learning. This work formally introduces the concept of “modality interference,” empirically demonstrating its detrimental impact on detection performance. To address this, we propose FND-MoE—a novel framework integrating Mixture-of-Experts (MoE), cross-modal feature alignment, and a dual-path adaptive feature selection mechanism to dynamically identify and suppress interfering modalities. Additionally, we introduce a modality importance reweighting strategy to enhance robustness. Evaluated on FakeSV and FVC-2018 benchmarks, FND-MoE achieves absolute accuracy improvements of 3.45% and 3.71%, respectively, surpassing state-of-the-art methods. Our approach establishes a new paradigm for robust multimodal fake news detection by explicitly modeling and mitigating modality-specific interference.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The rapid growth of social media has led to the widespread dissemination of fake news across multiple content forms, including text, images, audio, and video. Compared to unimodal fake news detection, multimodal fake news detection benefits from the increased availability of information across multiple modalities. However, in the context of social media, certain modalities in multimodal fake news detection tasks may contain disruptive or over-expressive information. These elements often include exaggerated or embellished content. We define this phenomenon as modality disruption and explore its impact on detection models through experiments. To address the issue of modality disruption in a targeted manner, we propose a multimodal fake news detection framework, FND-MoE. Additionally, we design a two-pass feature selection mechanism to further mitigate the impact of modality disruption. Extensive experiments on the FakeSV and FVC-2018 datasets demonstrate that FND-MoE significantly outperforms state-of-the-art methods, with accuracy improvements of 3.45% and 3.71% on the respective datasets compared to baseline models.

Problem

• Addressing modality disruption in multimodal fake news detection
• Proposing FND-MoE framework to mitigate disruptive modality effects
• Improving detection accuracy via two-pass feature selection mechanism

Innovation

• Proposes FND-MoE framework for fake news detection
• Introduces two-pass feature selection mechanism
• Addresses modality disruption in multimodal content