Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks

2025-07-14

AI Summary

Current XAI research for data analysis faces three key challenges: ambiguous task definitions, detachment from real-world usage contexts, and insufficient validation with target users—leading to scarce design guidelines and contradictory conclusions. To address these, we propose a novel “What–Why–Who” tri-dimensional classification framework, integrating insights from visual analytics, cognitive science, and dashboard design to establish the first interdisciplinary XAI taxonomy tailored to data analysis tasks. We further introduce a task-oriented framework modeling approach, user-role modeling strategies, and reusable guidelines for study design and reporting. Through a systematic literature review and empirical analysis, our work significantly enhances the comparability, reproducibility, and generalizability of XAI research, enabling robust identification of research gaps and facilitating consensus on effective design principles.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Web data visualization

Abstract

Research into explainable artificial intelligence (XAI) for data analysis tasks suffer from a large number of contradictions and lack of concrete design recommendations stemming from gaps in understanding the tasks that require AI assistance. In this paper, we drew on multiple fields such as visual analytics, cognition, and dashboard design to propose a method for categorising and comparing XAI studies under three dimensions: what, why, and who. We identified the main problems as: inadequate descriptions of tasks, context-free studies, and insufficient testing with target users. We propose that studies should specifically report on their users' domain, AI, and data analysis expertise to illustrate the generalisability of their findings. We also propose study guidelines for designing and reporting XAI tasks to improve the XAI community's ability to parse the rapidly growing field. We hope that our contribution can help researchers and designers better identify which studies are most relevant to their work, what gaps exist in the research, and how to handle contradictory results regarding XAI design.

Problem

• Inadequate descriptions of data analysis tasks in XAI studies
• Context-free studies lacking real-world application relevance
• Insufficient testing with target users in XAI research

Innovation

• Categorizing XAI studies using three dimensions
• Proposing study guidelines for XAI tasks
• Identifying gaps in XAI research and contradictions