Effects of Varying LLM Access on Essay Writing Behavior

2026-05-29

AI Summary

This study investigates the impact of varying levels of large language model (LLM) usage on undergraduate students’ writing performance, engagement, and sense of authorship. In a randomized controlled experiment, participants were assigned to one of three conditions: no LLM access, limited LLM use (≤3 prompts with responses ≤100 words), or unrestricted LLM access. Writing behaviors and learning outcomes were assessed through textual analysis and survey instruments. Results revealed no significant differences in essay quality across groups; however, the limited-use condition significantly enhanced students’ sense of authorship, organizational skills, and strategic revision practices. This work provides the first empirical evidence that moderate constraints on LLM usage can preserve the benefits of AI assistance while effectively fostering learner agency and higher-order writing strategies.

Abstract

Investigating the degree to which large language models (LLMs) affect teaching and learning in universities can help identify strategies for integrating LLMs in a way that supports, rather than undermines, student learning outcomes. This study examined how varying levels of LLM assistance affect writing performance, engagement, and perceived authorship. We report a pilot study in which 24 college students were randomly assigned to write a short essay with no LLM access, limited access (<=3 prompts, responses capped at 100 words), or unlimited access. Overall essay quality was statistically indistinguishable across groups. Yet writing behavior and perceived authorship diverged sharply: students with limited access reported higher ownership (62.5% would submit the essay as independent work, vs. 25% in the unlimited group), stronger organizational gains, and more strategic, revision-focused prompting. The unlimited group spent more time writing, produced essays more similar to LLM output, and reported reduced creative expression. Our findings suggest that constraining, rather than banning, LLM access may preserve authorship confidence while retaining the scaffolding benefits of AI assistance.

Problem

• large language models
• essay writing
• authorship
• student learning
• AI assistance

Innovation

• LLM access constraints
• writing behavior
• perceived authorship
• AI-assisted learning
• prompting strategy