contradiction detection

This work proposes a novel approach to open-domain claim verification that addresses the limitations of existing fact-checking systems, which often rely on a single knowledge source and thus fail to capture perspective divergence, resulting in limited coverage and poor transparency. The method leverages large language models (LLMs) to simultaneously retrieve multi-source evidence—such as from Wikipedia, PubMed, and Google—for both the original claim and its negation, integrating supporting and contradicting information. It further incorporates cross-source disagreement analysis to better model the complexity and diversity of information. By combining evidence deduplication, confidence scoring, and visualization, the approach is evaluated across four benchmark datasets using five distinct LLMs, achieving significant accuracy improvements and revealing notable differences in how various knowledge sources contribute to reasoning.

This work addresses the challenge faced by large language model (LLM) memory systems in retrieving the most up-to-date and accurate facts when confronted with temporally evolving contradictory information. To resolve this, the authors propose a deterministic memory conflict resolution method that explicitly identifies the bottleneck in the post-retrieval assembly phase. Instead of relying on LLM-based judgment, the approach employs deterministic aggregation strategies—such as max(serial) or max(timestamp)—after candidate fact extraction. Integrated with Self-Ask multi-hop reasoning and a unified retrieval chunking mechanism, the method forms an end-to-end optimized pipeline. Experimental results demonstrate significant improvements: achieving 78.0% accuracy with gpt-4o-mini and 94.8% with gpt-4o on the single-hop FactConsolidation task, and 51.5% on the multi-hop setting with gpt-4o, outperforming existing systems by 20–28 percentage points.

Existing metrics for factuality and faithfulness struggle to evaluate how language models handle documents containing both supporting and contradictory evidence. This work proposes ConflictScore, the first formal and quantitative framework for assessing a model’s ability to recognize and articulate conflicting evidence. It decomposes model responses into atomic claims, fine-grained labels their relationships with all source documents, and introduces two complementary metrics: CS-C (Conflict Sensitivity) and CS-R (Response Reasonableness). Built upon this framework, the ConflictBench benchmark encompasses diverse conflict types. Experiments demonstrate that ConflictScore effectively identifies overconfident claims across domains and serves as a feedback signal that significantly improves model truthfulness on TruthfulQA.

This study addresses structural conflicts—arising both among heterogeneous data sources and between data and model assumptions—in evidence synthesis models. We propose a general conflict detection framework based on score-based discrepancy measures. Methodologically, we extend prior–data conflict diagnostics to the latent space of hierarchical models, enabling inconsistency detection under multilevel and non-exchangeable structures; integrating Bayesian evidence synthesis, score-function-based metrics, and posterior simulation, our approach provides quantitative assessment of model assumption–data compatibility. Key contributions include: (1) moving beyond conventional bias diagnostics confined to the prior–likelihood level; (2) demonstrating high sensitivity to conflicts in both exchangeable and non-exchangeable models; and (3) exhibiting complementary diagnostic capability to existing methods in a real-world influenza severity model, thereby significantly enhancing the reliability of complex Bayesian inference.

In digital forensics, the atomicity and integrity of storage snapshots lack rigorous definitions that jointly guarantee both instantaneousness and causal ordering—undermining evidentiary admissibility in legal proceedings. To address this, we propose a novel atomicity definition grounded in causal consistency, overcoming the limitation of conventional time-based atomicity models. We further rectify conceptual flaws in existing integrity definitions and introduce a revised, theoretically sound yet engineering-practical integrity criterion—explicitly supporting copy-on-write (CoW) implementations. Our approach integrates causal modeling, formal snapshot semantics, CoW mechanism analysis, and formalization of forensic quality criteria, yielding a verifiable snapshot semantic framework. This work establishes the first theoretical foundation for forensic tool design that unifies causal ordering with instantaneous state capture, thereby significantly enhancing the forensic validity and judicial admissibility of live data acquisition.

Existing long-term memory systems lack effective evaluation of memory retrieval and ranking capabilities under conflicting scenarios, making it difficult to assess their reliability in terms of temporal validity, factual correctness, and contextual appropriateness. This work proposes MemConflict, a diagnostic framework that systematically defines and simulates three types of memory conflicts—dynamic, static, and conditional—and constructs a multi-turn dialogue benchmark incorporating cross-session conflicts and semantic interference. The framework enables both black-box answer evaluation and white-box memory analysis, revealing a notable disconnect between answer correctness and underlying memory quality. Experiments across six state-of-the-art systems demonstrate significant performance degradation under various conflict conditions, with pronounced sensitivity to conversation history length, distractor presence, implicit queries, and conflict distance.

This study addresses a critical issue in large language model (LLM) agents wherein tentative statements are erroneously solidified into definitive facts during memory compression, leading subsequent reasoning to rely uncritically on unverified information. The authors identify this “confidence fabrication” phenomenon as stemming from the linguistic certainty of phrasing rather than the reliability of the source itself. Through multi-turn dialogues, memory mechanism analysis, and adversarial memory injection, they demonstrate that a single misleading memory can induce erroneous judgments, yet introducing just one additional corroborating source suffices to restore accurate decision-making. Furthermore, conventional approaches such as trust labels or explicit negation instructions prove not only ineffective but potentially detrimental, exacerbating the problem rather than mitigating it.

Current evaluations of long-term memory in large language models predominantly rely on accuracy metrics for isolated questions, which fail to capture the dynamic behavior of user-specific facts under varying conditions. This work proposes MemTrace, a novel benchmark that shifts the evaluation unit from individual questions to knowledge points and constructs multidimensional, controllable probes along three axes: memory age, question type, and evidence condition. Using this framework, we systematically evaluate 13 system configurations across four memory paradigms and reveal that high overall accuracy masks significant deficiencies in tracking factual evolution and correcting erroneous premises. These failures primarily stem not from missing retrieval but from an inability to effectively leverage available evidential context.

Current evaluation frameworks struggle to characterize the arbitration behavior of large language models (LLMs) in retrieval-augmented generation (RAG)-based fact verification, particularly when conflicts arise between parametric priors and retrieved evidence. This work proposes PAVE, a diagnostic platform that introduces the first prior-aware verifier evaluation framework. It categorizes models into four cognitive states based on the correctness and confidence of their priors and quantifies their arbitration strategies using Jensen-Shannon divergence (JSD). Experiments across seven mainstream LLMs reveal that existing verifiers exhibit highly model-dependent and unreliable arbitration behavior. The proposed lightweight JSD-based arbitration method, which requires no model modification, significantly improves fact-checking accuracy and demonstrates robust performance across diverse model families.

This work addresses the challenge that existing personal AI agents struggle to provide reliable answers—or appropriately abstain—when confronted with multi-source, conflicting, or incomplete memories. The authors introduce the first diagnostic evaluation framework that systematically disentangles whether errors stem from poor evidence quality or flaws in conflict resolution mechanisms. They also release a large-scale, controllable benchmark dataset comprising 34,560 instances across 18 question templates and 8 reasoning types. By integrating structured methods with state-of-the-art large language models (LLMs), the best-performing model achieves 80.3% accuracy overall. When augmented with an abstention mechanism, it attains a selective accuracy of 85.3% at 78.3% coverage, substantially outperforming the strongest baseline LLM, which achieves only 71.0% accuracy at 95.4% coverage.