🤖 AI Summary
Existing research lacks a systematic, evidence-based understanding of frequent programming errors encountered by data engineering beginners. Method: We conducted a systematic literature review (SLR) of 21 studies published between 2003 and 2024, integrating general programming error theories with domain-specific characteristics of data engineering to develop the first empirically grounded, cross-language (Python/SQL/Java), cross-task error taxonomy. Using thematic coding and multi-source evidence synthesis, we structured the taxonomy along dimensions including syntax, logic, data modeling, and distributed semantics. Contribution/Results: Our framework breaks from traditional siloed analyses—by language or task—and establishes a unified, empirically validated, and extensible classification system. It provides actionable, evidence-based foundations for designing programming tools, enhancing IDE intelligent assistance (e.g., error detection and suggestion), and informing pedagogical interventions in data engineering education.
📝 Abstract
The design of effective programming languages, libraries, frameworks, tools, and platforms for data engineering strongly depends on their ease and correctness of use. Anyone who ignores that it is humans who use these tools risks building tools that are useless, or worse, harmful. To ensure our data engineering tools are based on solid foundations, we performed a systematic review of common programming mistakes in data engineering. We focus on programming beginners (students) by analyzing both the limited literature specific to data engineering mistakes and general programming mistakes in languages commonly used in data engineering (Python, SQL, Java). Through analysis of 21 publications spanning from 2003 to 2024, we synthesized these complementary sources into a comprehensive classification that captures both general programming challenges and domain-specific data engineering mistakes. This classification provides an empirical foundation for future tool development and educational strategies. We believe our systematic categorization will help researchers, practitioners, and educators better understand and address the challenges faced by novice data engineers.