Human-AI Experience in Integrated Development Environments: A Systematic Literature Review

2025-03-08

AI Summary

This study investigates the impact mechanisms and practical challenges of AI integration on Human-AI Interaction (HAX) within Integrated Development Environments (IDEs). Through a rigorous systematic literature review (SLR) of 89 high-quality studies, augmented by thematic coding and cross-study evidence synthesis, we establish the first comprehensive research framework for in-IDE HAX. Results reveal that while AI enhances development efficiency, it concurrently exacerbates novice dependency, verification overhead, and security vulnerabilities—primarily due to insufficient model explainability, lack of real-time validation, and absence of adaptive permission control. To address these limitations, we propose three novel research directions: longitudinal human-AI co-development studies, personalized interaction strategies, and AI governance mechanism design. Our findings provide empirical foundations and a technical roadmap for building trustworthy, controllable, and evolvable AI-augmented development environments.

Technology Category

• Humans and AI: Human-Computer Interaction
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Intelligent Robots: Human-Robot Interaction

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The integration of Artificial Intelligence (AI) into Integrated Development Environments (IDEs) is reshaping software development, fundamentally altering how developers interact with their tools. This shift marks the emergence of Human-AI Experience in Integrated Development Environment (in-IDE HAX), a field that explores the evolving dynamics of Human-Computer Interaction in AI-assisted coding environments. Despite rapid adoption, research on in-IDE HAX remains fragmented which highlights the need for a unified overview of current practices, challenges, and opportunities. To provide a structured overview of existing research, we conduct a systematic literature review of 89 studies, summarizing current findings and outlining areas for further investigation. Our findings reveal that AI-assisted coding enhances developer productivity but also introduces challenges, such as verification overhead, automation bias, and over-reliance, particularly among novice developers. Furthermore, concerns about code correctness, security, and maintainability highlight the urgent need for explainability, verification mechanisms, and adaptive user control. Although recent advances have driven the field forward, significant research gaps remain, including a lack of longitudinal studies, personalization strategies, and AI governance frameworks. This review provides a foundation for advancing in-IDE HAX research and offers guidance for responsibly integrating AI into software development.

Problem

• Explores Human-AI interaction in AI-assisted coding environments.
• Identifies challenges like verification overhead and automation bias.
• Highlights need for explainability and adaptive user control.

Innovation

• Systematic review of 89 AI-IDE integration studies
• Focus on explainability, verification, and user control
• Identifies gaps in longitudinal and personalized AI research