AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detection

2026-03-05

AI Summary

This study addresses the frequent conflation of conspiracy cue extraction and conspiracy belief detection in psycholinguistic research by proposing a decoupled framework that separately models semantic reasoning and structural localization to enhance both interpretability and accuracy. The core innovations include a Dynamic Discriminative Chain-of-Thought (DD-CoT) mechanism to mitigate semantic ambiguity and character-level fragility, and an “anti-echo chamber” architecture that effectively reduces misclassification of objective reporting as conspiratorial. The system integrates agent-based large language models, an adversarial parallel committee, and a calibrated adjudication mechanism. On the S1 task, it achieves a Macro F1 score of 0.24—representing a 100% improvement over the baseline—and 0.79 on the S2 task, a 49% gain, placing the S1 system third on the development set leaderboard.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper presents a novel agentic LLM pipeline for SemEval-2026 Task 10 that jointly extracts psycholinguistic conspiracy markers and detects conspiracy endorsement. Unlike traditional classifiers that conflate semantic reasoning with structural localization, our decoupled design isolates these challenges. For marker extraction, we propose Dynamic Discriminative Chain-of-Thought (DD-CoT) with deterministic anchoring to resolve semantic ambiguity and character-level brittleness. For conspiracy detection, an"Anti-Echo Chamber"architecture, consisting of an adversarial Parallel Council adjudicated by a Calibrated Judge, overcomes the"Reporter Trap,"where models falsely penalize objective reporting. Achieving 0.24 Macro F1 (+100\% over baseline) on S1 and 0.79 Macro F1 (+49\%) on S2, with the S1 system ranking 3rd on the development leaderboard, our approach establishes a versatile paradigm for interpretable, psycholinguistically-grounded NLP.

Problem

• psycholinguistic marker extraction
• conspiracy endorsement detection
• semantic ambiguity
• reporter trap
• structured localization

Innovation

• Agentic LLM
• Dynamic Discriminative Chain-of-Thought
• Anti-Echo Chamber
• Psycholinguistic Marker Extraction
• Conspiracy Endorsement Detection