Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models

2026-06-11

AI Summary

This study investigates how the mere presence of citations—regardless of their veracity—independently influences the reasoning behavior and hallucination propensity of large language models in citation-augmented settings. To this end, we introduce AuthorityBench, a benchmark employing a 2×2 factorial design that systematically crosses the truthfulness of claims with that of citations across four domains and 40 templates, while controlling for journal prestige and author nationality. Our fully balanced, multi-domain evaluation reveals that the presence of citations alone significantly increases hallucination rates by 3–22 percentage points, with the strongest effect observed when true claims are paired with fabricated citations. Hallucination rates range from 35% to 77% in general knowledge domains, whereas legal reasoning proves more robust; notably, neither journal prestige nor author nationality exerts a discernible influence.

Abstract

Large language models are increasingly deployed in citation-augmented settings, yet the effect of citation presence on model behavior independent of factual content remains poorly understood. We introduce AuthorityBench, a 220,564-prompt multi-domain benchmark that isolates how citation-based authority signals influence epistemic behavior in LLMs. The benchmark uses a fully balanced 2x2 factorial design crossing claim veracity with citation veracity, the first to do so, across four domains (general knowledge, science, law, and medicine), with controlled variation over 40 prompt templates, four venue prestige tiers, and a country-coded author name dataset. Evaluating seven models on 12 structured research questions, we find that citation presence, whether real or fabricated, consistently increases hallucination rates relative to a no-citation baseline. The effect is strongest when fabricated citations accompany true claims, raising hallucination rates by 3 to 22 percentage points and reaching 35 to 77% in the general knowledge domain, while legal claims are comparatively robust and venue prestige and author demographics show negligible impact. All datasets and evaluation code are available at: https://github.com/floating-reeds/AuthorityBench

Problem

• citation bias
• epistemic susceptibility
• large language models
• hallucination
• authority signals

Innovation

• citation bias
• epistemic susceptibility
• factorial benchmark design
• hallucination analysis
• authority signals