Detecting LLM-Generated Tokens in Human--LLM Coauthored Text

2026-07-23

AI Summary

This work addresses the limitation of existing methods that only support document-level detection and struggle to pinpoint specific segments generated by large language models (LLMs) within human–AI collaborative texts. The authors propose a token-level detection approach based on locally smoothed detection scores, employing an adaptive Lepski-type rule to select the smoothing bandwidth. This method effectively captures local authorship structure and, for the first time, enables fine-grained localization of LLM-generated content without requiring token-level annotations, while providing theoretical guarantees on estimation error. Experimental results demonstrate that the proposed approach significantly outperforms multiple baselines on both synthetic and real-world datasets, and it has been deployed on a publicly accessible detection website.

Abstract

The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents. Existing methods for detecting LLM-generated text mainly focus on document-level classification and cannot identify which parts of the text are generated by LLMs. This paper introduces a new method to address this urgent need. Our method operates at the token level, the natural unit of modern language models, and builds on existing token-level detection scores. The key idea is to smooth adjacent token scores to reduce their variability, while using an adaptive Lepski-type rule to select the bandwidth according to the local authorship structure. Our method is simple to implement and does not require token-level labeled data for training. Theoretically, we characterize this trade-off and show that the proposed method achieves favorable mean square error performance in estimating the underlying signal. Empirically, we demonstrate strong performance of our method against a wide range of baselines in both synthetic datasets and a realistic dataset. We deploy a publicly accessible website that implements the methods as well.

Problem

• LLM-generated text detection
• human-AI coauthorship
• token-level localization
• mixed-authorship documents
• fine-grained detection

Innovation

• token-level detection
• LLM-generated text
• adaptive smoothing
• Lepski-type bandwidth selection
• human-AI coauthorship