Evaluating the Reliability of Digital Forensic Evidence Discovered by Large Language Model: A Case Study

2025-07-08

Annual International Computer Software and Applications Conference

AI Summary

This study addresses the challenges of insufficient evidentiary reliability and lack of traceability in current AI-assisted digital forensics, particularly when large language models (LLMs) are involved, which often fail to meet judicial standards for trustworthiness. To overcome these limitations, the authors propose an automated framework that integrates LLMs with a Digital Forensics Knowledge Graph (DFKG), enabling end-to-end traceability of forensic data through deterministic unique identifiers (UIDs). The framework further incorporates a cross-validation mechanism to ensure both the integrity of the evidence chain and contextual consistency. Evaluated on a real-world 13 GB dataset, the approach achieves over 95% accuracy in forensic item extraction, establishing a novel, auditable, scalable, and legally compliant paradigm for AI-assisted digital investigations.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The growing reliance on AI-identified digital evidence raises significant concerns about its reliability, particularly as large language models (LLMs) are increasingly integrated into forensic investigations. This paper proposes a structured framework that automates forensic artifact extraction, refines data through LLM-driven analysis, and validates results using a Digital Forensic Knowledge Graph (DFKG). Evaluated on a 13 GB forensic image dataset containing 61 applications, 2,864 databases, and 5,870 tables, the framework ensures artifact traceability and evidentiary consistency through deterministic Unique Identifiers (UIDs) and forensic cross-referencing. We propose this methodology to address challenges in ensuring the credibility and forensic integrity of AI-identified evidence, reducing classification errors, and advancing scalable, auditable methodologies. A comprehensive case study on this dataset demonstrates the framework’s effectiveness, achieving over 95% accuracy in artifact extraction, strong support of chain-of-custody adherence, and robust contextual consistency in forensic relationships. Key results validate the framework’s ability to enhance reliability, reduce errors, and establish a legally sound paradigm for AI-assisted digital forensics.

Problem

• digital forensic evidence
• large language models
• evidentiary reliability
• forensic integrity
• AI-assisted forensics

Innovation

• Large Language Models
• Digital Forensic Knowledge Graph
• Artifact Traceability
• Deterministic UIDs
• AI-assisted Forensics