A Systematic Mapping Study on Chatbots in Programming Education

2025-09-09

AI Summary

This study addresses the absence of a systematic review on chatbot applications in programming education. Adopting a Systematic Mapping Study (SMS) methodology, it conducts multidimensional coding analysis across 54 empirical studies, examining bot types, pedagogical content, interaction modalities, and technical architectures. The work introduces the first comprehensive feature mapping framework specifically designed for chatbots in programming education. Findings reveal a pronounced concentration on introductory Python instruction and short-term task support, while identifying critical research gaps in longitudinal learning outcome evaluation, adaptive and personalized interaction mechanisms, and support for multilingual contexts or advanced programming concepts. The results provide empirically grounded design guidelines and strategic directions for advancing educational chatbot development—emphasizing scalability, personalization, and pedagogical depth—to better align artificial intelligence tools with evidence-based computing education practices.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Intelligent Robots: Other Foundations and Applications
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Educational chatbots have gained prominence as support tools for teaching programming, particularly in introductory learning contexts. This paper presents a Systematic Mapping Study (SMS) that investigated how such agents have been developed and applied in programming education. From an initial set of 3,216 publications, 54 studies were selected and analyzed based on five research subquestions, addressing chatbot types, programming languages used, educational content covered, interaction models, and application contexts. The results reveal a predominance of chatbots designed for Python instruction, focusing on fundamental programming concepts, and employing a wide variety of pedagogical approaches and technological architectures. In addition to identifying trends and gaps in the literature, this study provides insights to inform the development of new educational tools for programming instruction.

Problem

• Investigating chatbot development and application in programming education contexts
• Analyzing chatbot types, languages, content, and interaction models used
• Identifying trends and gaps for developing new programming education tools

Innovation

• Systematic mapping study methodology
• Python-focused chatbot instruction
• Diverse pedagogical technological architectures