Using CognitIDE to Capture Developers' Cognitive Load via Physiological Activity During Everyday Software Development Tasks

2025-03-05

AI Summary

This study addresses the need for unobtrusive, real-time assessment of developers’ cognitive load during authentic programming tasks to inform human-centered IDE optimization. Method: We designed and implemented CognitIDE, an IntelliJ plugin enabling end-to-end in-IDE cognitive load sensing—integrating multimodal physiological signals (electrodermal activity and heart rate variability), synchronizing them with fine-grained IDE events (e.g., code-line edits), and dynamically visualizing load fluctuations. A task-labeling and temporal synchronization mechanism supports uninterrupted, naturalistic data collection over up to one hour. Contribution/Results: Empirical evaluation confirms feasibility and user acceptability: physiological metrics significantly differentiate task-active from baseline states (p < 0.01); we achieve precise line-level cognitive load mapping and quantitative analysis. CognitIDE establishes a scalable, empirically grounded technical framework for cognitive-aware development environments—advancing both methodology and application in software engineering human factors research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Intelligent Robots: Cognitive Robotics
• Humans and AI: Brain-Sensing and Analysis

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Integrated development environments (IDE) support developers in a variety of tasks. Unobtrusively capturing developers' cognitive load while working on different programming tasks could help optimize developers' work experience, increase their productivity, and positively impact code quality. In this paper, we propose a study in which the IntelliJ-based IDE plugin CognitIDE is used to collect, map, and visualize software developers' physiological activity data while they are working on various software development tasks. In a feasibility study, participants completed four simulated everyday working tasks of software developers - coding, debugging, code documentation, and email writing - based on Java open source code in the IDE whilst their physiological activity was recorded. Between the tasks, the participants' perceived workload was assessed. Feasibility testing showed that CognitIDE could successfully be used for data collection sessions of one hour, which was the most extended duration tested and was well-perceived by those working with it. Furthermore, the recorded physiological activity indicated higher cognitive load during working tasks compared to baseline recordings. This suggests that cognitive load can be assessed, mapped to code positions, visualized, and discussed with participants in such study setups with CognitIDE. These promising results indicate the usefulness of the plugin for diverse study workflows in a natural IDE environment.

Problem

• Measure cognitive load during software development tasks.
• Optimize developer productivity and code quality.
• Visualize physiological activity data in IDE environments.

Innovation

• CognitIDE plugin captures physiological activity data.
• Data maps cognitive load to code positions.
• Visualizes cognitive load during software development tasks.