TROPIC -- Trustworthiness Rating of Online Publishers through online Interactions Calculation

2025-01-23

AI Summary

To address the high cost and poor scalability of news source credibility assessment, this paper proposes the first framework for modeling news outlet credibility based on large-scale social media user interaction behaviors (Twitter/X and Reddit). Methodologically, it integrates graph neural networks with interaction sequence modeling, incorporating collaborative signal mining and an explainable feedback mechanism to enable zero-shot initial evaluation for cold-start news sources and active-learning-driven iterative refinement. An interactive annotation platform is further developed to facilitate human verification. Experiments demonstrate cross-platform rating consistency of 86.4% (vs. expert panel), a 79.2% accuracy for cold-start initial evaluation, and a 3.8× improvement in human annotation efficiency. This work establishes a new paradigm for scalable, interpretable, and minimally human-dependent news credibility assessment.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Existing methods for assessing the trustworthiness of news publishers face high costs and scalability issues. The tool presented in this paper supports the efforts of specialized organizations by providing a solution that, starting from an online discussion, provides (i) trustworthiness ratings for previously unclassified news publishers and (ii) an interactive platform to guide annotation efforts and improve the robustness of the ratings. The system implements a novel framework for assessing the trustworthiness of online news publishers based on user interactions on social media platforms.

Problem

• News Source Reliability
• Cost-effective Method
• Publisher Credibility Assessment

Innovation

• Social Media Analysis
• News Credibility Assessment
• Interactive Platform Enhancement