TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

2026-03-20

AI Summary

This study addresses the emerging threat posed by AI-generated fake scientific tables to academic integrity, a challenge inadequately tackled by existing detection methods. The work presents the first systematic investigation of this issue, introducing FabTab—the first benchmark dataset comprising both AI-generated and human-authored scientific tables. The authors propose a novel detection approach centered on the perplexity discrepancy between a table’s structural skeleton and its numerical content. By integrating structural parsing, language model perplexity, and multi-perspective feature engineering within a Random Forest classifier, the method achieves strong performance, yielding AUROC scores of 0.987 and 0.883 on in-domain and out-of-domain evaluations, respectively, substantially outperforming current state-of-the-art techniques.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

AI-generated fabricated scientific manuscripts raise growing concerns with large-scale breaches of academic integrity. In this work, we present the first systematic study on detecting AI-generated fabricated scientific tables in empirical NLP papers, as information in tables serve as critical evidence for claims. We construct FabTab, the first benchmark dataset of fabricated manuscripts with tables, comprising 1,173 AI-generated papers and 1,215 human-authored ones in empirical NLP. Through a comprehensive analysis, we identify systematic differences between fabricated and real tables and operationalize them into a set of discriminative features within the TAB-AUDIT framework. The key feature, within-table mismatch, captures the perplexity gap between a table's skeleton and its numerical content. Experimental results show that RandomForest built on these features significantly outperform prior state-of-the-art methods, achieving 0.987 AUROC in-domain and 0.883 AUROC out-of-domain. Our findings highlight experimental tables as a critical forensic signal for detecting AI-generated scientific fraud and provide a new benchmark for future research.

Problem

• AI-generated tables
• scientific fraud
• academic integrity
• table detection
• fabricated manuscripts

Innovation

• AI-generated tables
• scientific fraud detection
• table authenticity
• perplexity mismatch
• FabTab benchmark