🤖 AI Summary
This study addresses the limited understanding of generalizable linguistic features that reliably distinguish human-written from large language model–generated text. Through large-scale empirical analysis, the authors evaluate the robustness of 284 interpretable linguistic features across 27 large language models and 10 textual domains, and develop a classifier driven solely by linguistic features. They identify lexical richness as the most robust discriminative signal across both models and domains—a finding reported for the first time—and expose the limitations of many context-dependent features. The results demonstrate that AI-generated text can be effectively detected using only linguistic cues, and pinpoint core features with strong generalization capabilities, thereby laying the groundwork for interpretable AI text detection.
📝 Abstract
Interpretable linguistic features offer a promising approach for explaining why a given text appears machine-generated, particularly for non-expert users. However, existing findings on which features reliably indicate LLM-generated text remain fragmented across feature sets, models, and text domains. To address this gap, we conduct a large-scale empirical study assessing the robustness of linguistic signals for characterizing AI-generated text. Our analysis covers 284 interpretable linguistic features across outputs from 27 LLMs and ten text domains under cross-model and cross-domain generalization settings. We show that classifiers based solely on linguistic features can reliably distinguish AI-generated from human-written text. However, many previously proposed indicators prove strongly context-dependent, with the exception of measures of lexical richness, which remain robust signals across model families and text domains. These results demonstrate which linguistic signals generalize across contexts and provide a foundation for more reliable, interpretable analyses of AI-generated language.