Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors

2025-07-08

AI Summary

To address catastrophic forgetting and insufficient environmental adaptability in continual multimodal misinformation detection (MMD), this paper proposes DAEDCMD—a Dirichlet-process-mixture-of-experts-based framework. It achieves event-level parameter isolation to mitigate forgetting, incorporates continuous-time dynamic modeling to capture societal environment evolution and enhance generalization to future data distributions, and unifies multimodal feature fusion with online learning—departing from conventional offline training paradigms. Evaluated on multiple benchmarks, DAEDCMD consistently outperforms six state-of-the-art multimodal detection baselines and three continual learning methods. Crucially, it preserves historical task performance while significantly improving detection accuracy on future-arriving data. To the best of our knowledge, DAEDCMD is the first framework to simultaneously achieve strong memory retention and high environmental adaptability in continual multimodal misinformation detection.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Nowadays, misinformation articles, especially multimodal ones, are widely spread on social media platforms and cause serious negative effects. To control their propagation, Multimodal Misinformation Detection (MMD) becomes an active topic in the community to automatically identify misinformation. Previous MMD methods focus on supervising detectors by collecting offline data. However, in real-world scenarios, new events always continually emerge, making MMD models trained on offline data consistently outdated and ineffective. To address this issue, training MMD models under online data streams is an alternative, inducing an emerging task named continual MMD. Unfortunately, it is hindered by two major challenges. First, training on new data consistently decreases the detection performance on past data, named past knowledge forgetting. Second, the social environment constantly evolves over time, affecting the generalization on future data. To alleviate these challenges, we propose to remember past knowledge by isolating interference between event-specific parameters with a Dirichlet process-based mixture-of-expert structure, and anticipate future environmental distributions by learning a continuous-time dynamics model. Accordingly, we induce a new continual MMD method DAEDCMD. Extensive experiments demonstrate that DAEDCMD can consistently and significantly outperform the compared methods, including six MMD baselines and three continual learning methods.

Problem

• Detect multimodal misinformation in evolving online data streams
• Overcome past knowledge forgetting in continual learning
• Improve future generalization amid changing social environments

Innovation

• Dirichlet process-based mixture-of-expert structure
• Continuous-time dynamics model learning
• Isolating interference between event-specific parameters