Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art

2026-07-31

AI Summary

This study addresses the challenges of automatic deception detection in legal contexts—namely, high domain sensitivity, data scarcity, and stringent interpretability demands. It presents the first systematic evaluation within a unified framework comparing fine-tuned Transformers and large language models (LLMs) across legal and general-domain deception detection tasks. The assessment encompasses six fine-tuned models, seven LLMs, four prompting strategies (including chain-of-thought), and seven datasets. Results indicate that fine-tuned models achieve superior performance in data-rich general domains, whereas few-shot LLMs remain competitive in low-resource legal settings. Notably, chain-of-thought prompting consistently underperforms direct classification. The findings underscore the critical roles of domain adaptability and prompt design in shaping model effectiveness for deception detection.

Abstract

Deception detection has critical implications for legal proceedings, law enforcement, and online security. Although human judgment is limited in accuracy and scalability, Natural Language Processing (NLP) offers a data-driven alternative. We present a survey and comparative analysis of NLP-based Automatic Deception Detection (ADD) focusing on the legal domain, reviewing the evolution from feature-based machine learning to Large Language Model (LLM) approaches. We conduct a unified empirical evaluation across seven datasets (two legal, five general-domain), comparing six fine-tuned transformer models and seven LLMs under four prompting strategies. The results show strong domain sensitivity, with fine-tuned models excelling in data-rich general domains and few-shot LLMs remaining competitive in low-resource legal settings. Chain-of-Thought prompting often underperforms direct classification. These findings highlight the need for domain adaptation and interpretable systems in high-stakes legal contexts.

Problem

• deception detection
• legal domain
• natural language processing
• domain adaptation
• automatic deception detection

Innovation

• Automatic Deception Detection
• Legal NLP
• Large Language Models
• Domain Adaptation
• Prompting Strategies