EditLens: Quantifying the Extent of AI Editing in Text

2025-10-03

AI Summary

This work addresses the challenge of detecting and quantifying AI-assisted text editing—a task where existing methods lack continuous, fine-grained measurement capabilities. We propose EditLens, the first method enabling continuous quantification of AI editing intensity. Our approach introduces a lightweight text similarity metric as an intermediate supervisory signal and trains a regression model to predict the degree of AI modification applied to human-written text. The model simultaneously supports binary (human vs. AI-generated) and ternary (human vs. fully AI-generated vs. AI-edited) provenance classification. Evaluated on a large-scale manually annotated dataset, EditLens achieves 94.7% F1-score for binary classification and 90.4% for ternary classification—substantially outperforming state-of-the-art detectors. By enabling interpretable, continuous measurement of editing intensity, EditLens establishes a deployable technical foundation for authorship attribution, academic integrity assessment, and AI writing policy formulation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

A significant proportion of queries to large language models ask them to edit user-provided text, rather than generate new text from scratch. While previous work focuses on detecting fully AI-generated text, we demonstrate that AI-edited text is distinguishable from human-written and AI-generated text. First, we propose using lightweight similarity metrics to quantify the magnitude of AI editing present in a text given the original human-written text and validate these metrics with human annotators. Using these similarity metrics as intermediate supervision, we then train EditLens, a regression model that predicts the amount of AI editing present within a text. Our model achieves state-of-the-art performance on both binary (F1=94.7%) and ternary (F1=90.4%) classification tasks in distinguishing human, AI, and mixed writing. Not only do we show that AI-edited text can be detected, but also that the degree of change made by AI to human writing can be detected, which has implications for authorship attribution, education, and policy. Finally, as a case study, we use our model to analyze the effects of AI-edits applied by Grammarly, a popular writing assistance tool. To encourage further research, we commit to publicly releasing our models and dataset.

Problem

• Detecting AI-edited text using similarity metrics and regression models
• Quantifying the degree of AI modifications in human-written text
• Distinguishing human, AI-generated, and mixed authorship in writing

Innovation

• Lightweight similarity metrics quantify AI editing magnitude
• Regression model predicts AI editing amount using metrics
• Model achieves state-of-the-art classification performance