Beyond Productivity: Measuring Developers' Cognitive Load During GenAI-Supported Software Development

2026-09-29

AI Summary

This study addresses the predominant focus on productivity in existing research on generative AI (GenAI)-assisted software development, which largely overlooks developers’ cognitive load. We propose a developer-centric evaluation framework and conduct an industrial field study utilizing EmbracePlus wristbands to collect physiological data, complemented by self-reported logs and questionnaires, to systematically analyze how GenAI usage and task contexts influence perceived cognitive load. Our findings reveal that cognitive load is significantly modulated by GenAI interaction patterns and task contexts. Furthermore, we demonstrate that physiological signals are most effective as supplements to subjective reports rather than standalone metrics. By addressing the limitations of efficiency-only evaluations, this work provides empirical evidence for designing human-centered GenAI-assisted development tools.

Abstract

Generative AI (GenAI) is changing software development workflows and how developers work. Industry evaluations of GenAI adoption often monitor productivity gains, usage, and output quality, but limited attention is paid to the interaction experience and cognitive load of the actual adopters and drivers of GenAI technology - the software developers. Understanding whether GenAI changes or shifts developers' cognitive demands during everyday development is important for a developer-centered evaluation of GenAI-supported software development. It can inform organizations in designing and evaluating effective AI-supported workflows. In this work, we study how GenAI use and task context relate to professional developers' perceived cognitive load and whether wearable-derived physiological characteristics provide additional information beyond this context. In a four-day industrial field study at two SAP sites, 21 developers documented their tasks, task duration, GenAI use, and perceived cognitive load while wearing an EmbracePlus wristband. The results show that perceived cognitive load is associated with both GenAI use and task context, while physiological measures provide only limited additional information. These findings suggest that developers' perceived cognitive load during GenAI-supported software development should be evaluated in relation to the concrete work context, with wearable physiological data used as complementary rather than standalone information.

Problem

• Generative AI
• Cognitive Load
• Software Development
• Wearable Sensors
• Developer Experience

Innovation

• Cognitive Load
• Generative AI
• Wearable Sensors
• Software Development
• Physiological Measures