VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

2026-09-28

AI Summary

This study addresses the verification cost asymmetry and prioritization challenges arising from LLM-generated misinformation under resource constraints by constructing a unified benchmark to evaluate its verification complexity. Methodologically, it proposes multidimensional verification complexity metrics and a composite VEX scoring framework to quantify the verification resources consumed by information, employing LLM-as-judge, content analysis, and fact-checking agents for scalable evaluation. The findings reveal that no single method can encompass all verification dimensions. Furthermore, the study demonstrates that LLMs can generate high-VEX misinformation at negligible cost in bulk, which tends to be prioritized during screening and thereby introduces significant risks of verification resource misallocation.

Abstract

Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosystem. Under tight time, labor, and budget constraints, media organizations, platforms, and fact-checkers rely on screening to prioritize which content to verify. We introduce VEX-Bench, a unified benchmark for evaluating the verification complexity of LLM-generated misinformation, as perceived during screening, across models and generation methods. Verification complexity is assessed along multiple dimensions derived from journalistic and fact-checking practices, capturing checkability, harm potential, source credibility signals, imposter legitimacy, and expected verification effort. We define the VEX score as an integrated measure combining elicitation yield and verification complexity to quantify how generated content consumes limited verification capacity. We construct a benchmark spanning two misinformation categories, 6 high-stakes domains, and 60 real-world topics, and evaluate 7 frontier LLMs and 7 generation methods, yielding 5{,}880 articles. We employ an LLM-as-judge for scalable evaluation and validate it using content-analysis methodology, including ordinal Krippendorff $\alpha$ for inter-annotator reliability, complemented by fact-checking agents for verification. Our findings show that no single method dominates all dimensions, underscoring the need for multi-dimensional evaluation. LLMs can generate high-VEX misinformation at 3$\times$ to 169$\times$ lower cost than agent-based verification. Such content is often prioritized during screening, consuming scarce verification resources and introducing a systematic risk of misallocation in resource-constrained verification systems. The code is publicly available in our \href{https://github.com/HanxunH/VEX-Bench}{GitHub repository}.

Problem

• misinformation
• verification complexity
• large language models
• fact-checking
• benchmark

Innovation

• Verification Complexity
• VEX-Bench
• Misinformation
• LLM-as-Judge
• VEX Score