(Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding

2026-07-28

AI Summary

This study investigates whether code-generating agents, while enhancing development efficiency, inadvertently impair users’ deep understanding of the code they produce, thereby affecting their debugging, learning, and collaboration capabilities. In a controlled experiment, participants completed a web development task using either a code-generating agent or a conversational chatbot, followed by assessments of code comprehension and an unaided code extension task. The study provides the first empirical evidence that, although such agents significantly accelerate initial development, users exhibit shallower comprehension of the generated code and struggle to independently extend its functionality. Notably, low-engagement interaction patterns are strongly associated with diminished understanding. Despite reporting weaker self-assessed comprehension, participants consistently preferred using the agent due to its convenience and efficiency.

Abstract

Coding agents (e.g., Cursor) improve developer productivity by optimizing task completion, but shifting users from writing code to prompting and reviewing may harm their understanding, impeding oversight, learning, and communication. To probe this, we have 54 students create a website with one of two AI systems: an agent that edits user code; or a chatbot where users write code alone or adapt generic code snippets. We test understanding via comprehension questions and a task where users extend their code without agents, showing: (1) While agents aid initial task completion, they harm users' code comprehension and thus do not prepare users to extend their code; (2) Low-effort agent interaction types, like copy+paste prompts and auto-accepted edits, are linked with lower comprehension; and (3) Despite self-reported weaker understanding, users still prefer coding agents because they are quick and easy to use. While users stay in the loop for coding workflows, understanding should not be forgotten. Towards this goal, we distill our analyses into future research directions for coding agent developers: dissuading low-effort prompting, creating readable code, and promoting active engagement.

Problem

• coding agents
• code comprehension
• developer productivity
• AI-assisted programming
• human-AI collaboration

Innovation

• coding agents
• code comprehension
• human-AI collaboration
• developer productivity
• active engagement