HALLMARK: Diagnosing Three Failure Modes in LLM Citation Verifiers

2026-07-20

AI Summary

This work addresses the absence of a unified benchmark for diagnosing failure modes of large language model (LLM) citation verifiers in detecting fabricated references. We introduce HALLMARK, a comprehensive benchmark encompassing 14 hallucination categories, three difficulty levels, and six diagnostic subtests, comprising 2,526 BibTeX entries alongside a contamination-resistant held-out test set. Through systematic evaluation of diverse verification approaches—including DOI lookup, zero-shot LLM reasoning, tool-augmented agents, and our novel rule-based bibtex-updater—we uncover three primary failure modes for the first time. Our analysis reveals that high false positive rates constitute the main bottleneck for real-world deployment; notably, only the two most recently trained models maintain low false positive rates on out-of-distribution data, underscoring the critical importance of false positive control for practical feasibility.

Abstract

Large language models (LLMs) now routinely draft literature reviews and assist with academic writing, which means a higher risk of fabricated references: GPTZero found 53 papers with hallucinated citations among NeurIPS 2025's accepted set. Rule- and LLM-based verifiers are emerging, but no shared benchmark compares them and gives detailed failure diagnostics. We close that gap with HALLMARK (Hallucination benchmark): 2,526 BibTeX entries spanning 14 hallucination types, three difficulty tiers, six diagnostic sub-tests per entry, and a contamination-resistant held-out split. On it we evaluate a DOI-lookup baseline, frontier LLMs zero-shot, tool-augmented agents, and our own rule-based, co-designed verifier bibtex-updater. Across the benchmark one result is consistent: the false-positive rate, not recall, decides whether a verifier is deployable. HALLMARK makes it concrete through three failure modes: agentic lookups buy recall but inflate false positives; at a venue-realistic base rate, the order-of-magnitude spread in false-positive rates (FPRs) -- not recall -- governs whether a verifier's flags are mostly true catches or mostly noise; and most LLMs over-flag papers published past their training cutoff, where only the two latest-cutoff models hold their false-positive rate near in-distribution levels (a signal we report as descriptive, since it is confounded with possible recall of those entries). Thus FPR is the deployment bottleneck, but an undetected fabrication remains the costlier error for the scientific record.

Problem

• citation hallucination
• LLM verifiers
• benchmark
• false-positive rate
• failure modes

Innovation

• citation hallucination
• false-positive rate
• benchmarking
• LLM verifiers
• HALLMARK