Avoiding Over-smoothing in Social Media Rumor Detection with Pre-trained Propagation Tree Transformer

2026-03-24

AI Summary

This work addresses the limitations of graph neural networks (GNNs) in social media rumor detection, particularly their susceptibility to over-smoothing and difficulty in capturing long-range dependencies within propagation tree structures. To overcome these challenges, the authors propose P2T3, the first purely Transformer-based pre-trained model for propagation trees. P2T3 extracts conversational chains along propagation directions, incorporates token-level embeddings to encode relational connections, and injects inductive bias to guide learning. Trained via self-supervision on large-scale unlabeled data, this approach eliminates reliance on conventional GNNs, effectively mitigating over-smoothing while enabling few-shot learning and multimodal extensions. Extensive experiments demonstrate that P2T3 significantly outperforms state-of-the-art methods across multiple benchmark datasets, with especially strong performance in low-resource settings.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Deep learning techniques for rumor detection typically utilize Graph Neural Networks (GNNs) to analyze post relations. These methods, however, falter due to over-smoothing issues when processing rumor propagation structures, leading to declining performance. Our investigation into this issue reveals that over-smoothing is intrinsically tied to the structural characteristics of rumor propagation trees, in which the majority of nodes are 1-level nodes. Furthermore, GNNs struggle to capture long-range dependencies within these trees. To circumvent these challenges, we propose a Pre-Trained Propagation Tree Transformer (P2T3) method based on pure Transformer architecture. It extracts all conversation chains from a tree structure following the propagation direction of replies, utilizes token-wise embedding to infuse connection information and introduces necessary inductive bias, and pre-trains on large-scale unlabeled datasets. Experiments indicate that P2T3 surpasses previous state-of-the-art methods in multiple benchmark datasets and performs well under few-shot conditions. P2T3 not only avoids the over-smoothing issue inherent in GNNs but also potentially offers a large model or unified multi-modal scheme for future social media research.

Problem

• rumor detection
• over-smoothing
• propagation tree
• graph neural networks
• long-range dependencies

Innovation

• Propagation Tree Transformer
• Over-smoothing Avoidance
• Pre-trained Rumor Detection
• Conversation Chain Modeling
• Few-shot Social Media Analysis