DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing

2026-09-28

AI Summary

This study addresses the challenge of inferring AI-assisted writing processes solely from final texts by developing a multi-view analytical environment that jointly captures textual, procedural, and AI-interaction signals to reconstruct document evolution. Methodologically, it innovatively integrates product, process, and interaction perspectives through multi-view data collection, longitudinal tracking analysis, and large language model interaction log mining, effectively identifying anomalous behaviors such as copy-pasting. Empirical evaluations demonstrate that this framework accurately identifies student writing patterns and generates actionable instructional analytics. Its interpretability has received strong endorsement from educators, establishing a novel paradigm for writing assessment in the era of artificial intelligence.

Abstract

Generative AI has changed how students produce writing assignments. The final artifact is no longer sufficient to understand the process through which it was produced. We introduce DraftTrace, a writing environment that jointly captures three complementary views of writing: the final product, the writing process and interactions with an integrated AI-assistant. DraftTrace reconstructs how a document develops over time and organizes these signals into submission, longitudinal, and class-level analytics for instructors. We deployed DraftTrace in a graduate NLP course with 81 students and compared their sessions with LLM-generated responses entered by automated tools and with copy-typed responses. While product measures distinguish differences in text formulation, process measures distinguish differences in how text is entered. Considering both views together helps characterize cases such as copy-typing. Interaction traces show that students use the assistant differently across stages of writing: to clarify the question at an early stage and to verify answers at a later stage. A preliminary instructor survey highlights the importance of multi-view writing analytics and their interpretability.

Problem

• Generative AI
• Writing analytics
• Writing process
• AI-assisted writing
• Multi-view analysis

Innovation

• Multi-View Analytics
• AI-Integrated Writing
• Writing Process Mining
• Interaction Traces
• Generative AI