Predicting Human Visual Attention on Words in Source Code

2026-07-15

AI Summary

This study addresses the challenge of accurately predicting visual attention distributions during source code reading to support development environment optimization and cognitive modeling. To this end, we propose a novel attention alignment loss function that explicitly aligns eye-tracking data with the internal attention mechanisms of neural networks. Our approach is the first to significantly outperform both large language models—such as Claude and GPT-5—and traditional baselines on code comprehension tasks. Experimental results across three Java and C datasets demonstrate substantial improvements in visual attention prediction, with Pearson correlation coefficients increasing by 64%, 16%, and 467%, respectively. Furthermore, our method achieves markedly superior performance in scanpath prediction compared to existing approaches.

Abstract

This paper presents a computational model to predict human visual attention over words in software source code. The visual attention of software engineers when reading source code has long been studied as a means to understand human cognitive processes during software engineering tasks. Predicting this visual attention is important for perfecting user interface design and understanding what information human programmers need. We propose a model of programmer visual attention in which we design a novel loss function that computes similarity between human attention measured during eye tracking experiments and the internal attention of the artificial neural network. We evaluate our model by comparing its outputs to actual eye tracking data from three separate datasets. Two are in the Java programming language and one is in the C programming language. Our model outperforms the baseline in software engineering by 64%, 16%, and 467% in each of these studies according to Pearson correlation. We used scanpath prediction as an example to demonstrate that our model is more capable of the task that requires the understanding of human thought process. Our model achieves a statistically significant improvement over the close baseline in the reading task according to normalized Levenshtein distance and outperforms both Claude and GPT-5 on both reading and writing tasks.

Problem

• visual attention
• source code
• eye tracking
• program comprehension
• human cognition

Innovation

• visual attention prediction
• source code comprehension
• eye tracking
• neural attention modeling
• scanpath prediction