fact verification

Existing metrics for factuality and faithfulness struggle to evaluate how language models handle documents containing both supporting and contradictory evidence. This work proposes ConflictScore, the first formal and quantitative framework for assessing a model’s ability to recognize and articulate conflicting evidence. It decomposes model responses into atomic claims, fine-grained labels their relationships with all source documents, and introduces two complementary metrics: CS-C (Conflict Sensitivity) and CS-R (Response Reasonableness). Built upon this framework, the ConflictBench benchmark encompasses diverse conflict types. Experiments demonstrate that ConflictScore effectively identifies overconfident claims across domains and serves as a feedback signal that significantly improves model truthfulness on TruthfulQA.

This work proposes a novel approach to open-domain claim verification that addresses the limitations of existing fact-checking systems, which often rely on a single knowledge source and thus fail to capture perspective divergence, resulting in limited coverage and poor transparency. The method leverages large language models (LLMs) to simultaneously retrieve multi-source evidence—such as from Wikipedia, PubMed, and Google—for both the original claim and its negation, integrating supporting and contradicting information. It further incorporates cross-source disagreement analysis to better model the complexity and diversity of information. By combining evidence deduplication, confidence scoring, and visualization, the approach is evaluated across four benchmark datasets using five distinct LLMs, achieving significant accuracy improvements and revealing notable differences in how various knowledge sources contribute to reasoning.

This work addresses the critical issue of “superficially plausible yet unsupported claims” in large language model (LLM)-generated scientific paper reviews. To tackle claim grounding—the alignment between review weaknesses and explicit claims in the paper—we introduce CLAIMCHECK, the first benchmark dedicated to scientific claim alignment evaluation. Built upon NeurIPS 2023/2024 submissions and official reviews, CLAIMCHECK features fine-grained expert annotations mapping review weaknesses to original claims, assessing their validity, objectivity, and type. Based on this, we define three novel tasks: weakness–claim linkage, weakness rewriting, and claim verification. Our methodology integrates OpenReview data mining, expert human annotation, and multi-task LLM evaluation (matching, classification, generation, and reasoning-based verification). Experiments reveal that state-of-the-art LLMs significantly underperform humans on linkage and verification tasks—while achieving only moderate accuracy on weakness labeling—highlighting claim grounding as a fundamental bottleneck in automating scientific peer review.

This work addresses the covert fallacy of “misquoting biomedical literature” in health-related misinformation—where scientific sources are superficially cited but systematically misrepresented to support false claims. We introduce MissciPlus, the first benchmark dataset comprising real-world misquotation passages paired with corresponding false assertions, covering diverse logical fallacies. Methodologically, we integrate retrieval models, large language model (LLM)-based reasoning, and evidence-level fact-checking to jointly detect fallacies, localize misleading passages, and provide interpretable attributions. Experiments reveal that existing fact-checking models largely fail to identify misquotation-based fallacies, and that misquotation significantly increases LLMs’ credulity toward false claims. This study establishes the first logic-fallacy benchmark grounded in authentic scientific misuse scenarios, advancing evidence-aware fact-checking research focused on source credibility and interpretability.

Existing methods for detecting out-of-context (OOC) misinformation—where authentic images are falsely paired with misleading textual claims—largely rely on external evidence but neglect the evidential stance (supportive or refuting) toward the claim. Method: We propose a stance-aware multimodal detection framework featuring: (i) a novel Stance Extraction Network (SEN) that models evidential stance as a critical deviation signal; (ii) fine-grained semantic alignment via named-entity co-occurrence–based support-refute scoring; and (iii) an end-to-end trainable architecture integrating joint text-image encoding. Contribution/Results: Evaluated on large-scale public benchmarks, our approach surpasses state-of-the-art methods, achieving up to a 3.2% absolute accuracy gain. It significantly improves both robustness and interpretability in OOC misinformation detection by explicitly modeling and leveraging stance information.

This work addresses the issue of factual inconsistency between generated answers and cited sources in retrieval-augmented generation (RAG) systems by proposing a corrective RAG pipeline that integrates pre-generation passage filtering with post-generation strict entailment verification. Building upon Corrective RAG and CiteFix mechanisms, the approach further incorporates an LLM-as-judge diagnostic method to enhance citation fidelity and factual grounding. The proposed framework effectively improves the faithfulness of citations while preserving answer relevance and fluency, thereby demonstrating the feasibility of strengthening citation integrity in RAG outputs. Moreover, the study advocates for a new evaluation paradigm that prioritizes strict answer traceability to source evidence, emphasizing the need for more rigorous assessment of attribution accuracy in generative retrieval systems.

This study addresses the problem of "deceptive grounding" in clinical retrieval-augmented generation (RAG) systems, wherein models erroneously attribute genuine evidence for one drug to another, producing responses that appear plausible yet are substantively misleading. The work provides the first formal definition and quantitative assessment of this issue, introducing a controlled-factor benchmark to systematically evaluate 13 models across 740 drug–disease pairs. Surprisingly, domain-finetuned models exhibit higher error rates. To mitigate this, the authors propose a high-precision entity attribution verification mechanism combining inverse probability weighting correction with human-curated gold standards, achieving 97.0% precision and 98.7% recall under adversarial conditions. Real-world evaluation reveals an overall deceptive grounding rate of 7.8% in deployed systems—rising to 13.6% for novel drugs—highlighting both the severity of the problem and the efficacy of the proposed solution.

This work addresses the reliability challenges of Retrieval-Augmented Generation (RAG) in biomedical and clinical question answering by proposing a novel RAG framework that integrates hybrid retrieval, re-ranking, and claim-level evidence verification. The system leverages Amazon Bedrock for document processing and retrieval, enhances evidence relevance through Amazon Titan Text Embeddings V2, OpenSearch Serverless, and Cohere re-ranking, and employs a dedicated judgment model to rigorously verify each generated claim against source evidence. Evaluated on 25 biomedical queries, the framework successfully grounded all 200 extracted factual claims in their original evidence sources out of 500 retrieved passages, achieving 100.0% grounding accuracy and substantially improving the factual reliability and verifiability of generated responses.

This work addresses the challenge of automatically generating fact-checking articles grounded in verifiable citations by leveraging claims, veracity labels, and supporting evidence documents. To this end, the authors propose a multi-agent collaborative pipeline that integrates dense retrieval, source-balanced evidence selection, structured content planning, and citation-aware generation. The framework innovatively incorporates a gated self-evaluation mechanism and a natural language inference (NLI)-driven citation auditing module to repair missing citations and automatically eliminate redundant or unsupported references. Experimental results demonstrate that the proposed approach significantly improves citation accuracy and source credibility in the generated articles, thereby validating the effectiveness of jointly optimizing evidence selection, structured generation, and post-hoc citation verification.