GazeCopilot: Evaluating Novel Gaze-Informed Prompting for AI-Supported Code Comprehension and Readability

2025-11-11

AI Summary

Current AI coding assistants (e.g., GitHub Copilot) rely on static context, failing to adapt to developers’ real-time cognitive states—leading to overly generic prompts and excessive code refactoring. This work introduces the first dynamic prompt optimization method grounded in real-time eye-tracking feedback: it integrates gaze metrics—including fixation trajectories and pupil dilation—to construct an attention-aware closed-loop interaction system that delivers personalized code suggestions and demand-driven refactoring. Our core contribution lies in incorporating fine-grained cognitive state modeling into prompt engineering, thereby avoiding one-size-fits-all interventions. Empirical evaluation demonstrates that our approach significantly improves code comprehension accuracy (+23.6%), reduces comprehension time (−31.4%), and enhances subjective readability ratings (p < 0.01), outperforming both standard Copilot and predefined-strategy baselines.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Prompt Engineering / Prompting
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

AI-powered coding assistants, like GitHub Copilot, are increasingly used to boost developers'productivity. However, their output quality hinges on the contextual richness of the prompts. Meanwhile, gaze behaviour carries rich cognitive information, providing insights into how developers process code. We leverage this in Real-time GazeCopilot, a novel approach that refines prompts using real-time gaze data to improve code comprehension and readability by integrating gaze metrics, like fixation patterns and pupil dilation, into prompts to adapt suggestions to developers'cognitive states. In a controlled lab study with 25 developers, we evaluated Real-time GazeCopilot against two baselines: Standard Copilot, which relies on text prompts provided by developers, and Pre-set GazeCopilot, which uses a hard-coded prompt that assumes developers'gaze metrics indicate they are struggling with all aspects of the code, allowing us to assess the impact of leveraging the developer's personal real-time gaze data. Our results show that prompts dynamically generated using developers'real-time gaze data significantly improve code comprehension accuracy, reduce comprehension time, and improve perceived readability compared to Standard Copilot. Our Real-time GazeCopilot approach selectively refactors only code aspects where gaze data indicate difficulty, outperforming the overgeneralized refactoring done by Pre-set GazeCopilot by avoiding revising code the developer already understands.

Problem

• Enhancing code comprehension accuracy through real-time gaze data integration
• Reducing code comprehension time with dynamic gaze-informed prompt refinement
• Improving perceived readability by selectively refactoring problematic code segments

Innovation

• Integrates real-time gaze data into prompts
• Uses fixation patterns and pupil dilation metrics
• Dynamically refactors code based on cognitive states