Analyzing Students' Statistics Writing Before and After the Emergence of Large Language Models

2026-06-21

AI Summary

This study investigates whether the widespread adoption of large language models (LLMs) has led undergraduate students to overrely on generative AI in statistical writing, thereby affecting their statistical reasoning and communicative competence. Analyzing over 1,600 student data analysis reports from 2021 to 2025 through text similarity metrics, stylometric analysis, and large-scale corpus comparisons, this work provides the first empirical evidence of LLMs’ differential impact across report sections—most pronounced in introductions and conclusions. Findings indicate that student writing increasingly converges toward LLM-generated styles, particularly in these segments, while simultaneously aligning more closely with expert statistical discourse. This dual trend suggests that LLMs entail both cognitive risks and pedagogical potential. The study further proposes a novel assessment framework integrating statistical thinking to better evaluate AI-mediated learning outcomes.

Abstract

The ability to communicate statistical results to domain experts and stakeholders is an important goal of the undergraduate statistics and data science curriculum. However, as large language models (LLMs) have become more accessible, a major concern is that students are offloading important cognitive tasks to generative AI. Using a corpus of over 1,600 undergraduate students' data analysis reports from 2021 to 2025, we show how students' writing style and verb usage have become more similar to that of LLMs. This shift is most pronounced in the first and fifth quintiles of students' reports, which roughly map onto the introduction and conclusion sections, respectively. At the same time, we demonstrate that students' writing style has become more similar to that of statistics experts with the addition of LLMs. We end by discussing the implications of our findings for statistics and data science educators. In particular, we propose alternative modes of assessment that still emphasize statistical thinking, such as targeted writing assignments for structuring a report introduction.

Problem

• large language models
• statistics education
• student writing
• generative AI
• statistical communication

Innovation

• large language models
• statistical writing
• writing style analysis
• undergraduate education
• statistical thinking