AI Generated Text Detection Using Instruction Fine-tuned Large Language and Transformer-Based Models

2025-07-07

AI Summary

Existing methods struggle to simultaneously achieve high accuracy in distinguishing human-written text from AI-generated text and precisely identifying the specific large language model (LLM) responsible for generation (e.g., GPT-4o-mini, LLaMA-3, BERT). Method: We propose a unified discriminative framework that jointly performs AI-text detection and LLM attribution via instruction-tuned fine-tuning of three heterogeneous models—GPT-4o-mini, LLaMA-3-8B, and BERT—within a single end-to-end pipeline. Contribution/Results: Our approach introduces cross-architecture collaborative training and tightly coupled multi-task learning, enhancing robustness under complex, real-world conditions. Experiments demonstrate 95.47% accuracy in AI-text detection and 46.98% accuracy in fine-grained LLM attribution—substantially outperforming prior baselines. This framework provides a scalable, technically grounded solution for combating disinformation and ensuring AI content traceability.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)Humans and AI: Human-in-the-loop Machine Learning

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationEconomics, Online Markets and Human Computation: LLM based quality controls for crowd workUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) possess an extraordinary capability to produce text that is not only coherent and contextually relevant but also strikingly similar to human writing. They adapt to various styles and genres, producing content that is both grammatically correct and semantically meaningful. Recently, LLMs have been misused to create highly realistic phishing emails, spread fake news, generate code to automate cyber crime, and write fraudulent scientific articles. Additionally, in many real-world applications, the generated content including style and topic and the generator model are not known beforehand. The increasing prevalence and sophistication of artificial intelligence (AI)-generated texts have made their detection progressively more challenging. Various attempts have been made to distinguish machine-generated text from human-authored content using linguistic, statistical, machine learning, and ensemble-based approaches. This work focuses on two primary objectives Task-A, which involves distinguishing human-written text from machine-generated text, and Task-B, which attempts to identify the specific LLM model responsible for the generation. Both of these tasks are based on fine tuning of Generative Pre-trained Transformer (GPT_4o-mini), Large Language Model Meta AI (LLaMA) 3 8B, and Bidirectional Encoder Representations from Transformers (BERT). The fine-tuned version of GPT_4o-mini and the BERT model has achieved accuracies of 0.9547 for Task-A and 0.4698 for Task-B.

Problem

• Detect AI-generated text to prevent misuse in phishing and fake news.
• Identify specific LLM models responsible for generating the text.
• Improve detection accuracy using fine-tuned GPT, LLaMA, and BERT models.

Innovation

• Instruction fine-tuned GPT_4o-mini for detection
• Utilized LLaMA 3 8B for model identification
• Fine-tuned BERT for distinguishing AI-generated text