🤖 AI Summary
Visualization practitioners often lack formal training, resulting in significant gaps in design knowledge. Method: This paper presents the first systematic evaluation of large language models (LLMs) for visualization design consultation, employing a dual-path approach: quantitative analysis (multidimensional comparison of ChatGPT and expert responses from VisGuides) and qualitative analysis (double-blind user studies, content coding, and human-AI co-feedback analysis). Results: While ChatGPT rapidly generates diverse design alternatives, it substantially underperforms human experts in deep contextual understanding, visual intent inference, and support for nonlinear interactions. The study proposes a novel “context-enhanced + interaction-guided” paradigm tailored to design feedback, establishing both theoretical foundations and a practical framework for developing trustworthy LLM-assisted visualization design systems.
📝 Abstract
Data visualization creators often lack formal training, resulting in a knowledge gap in design practice. Large language models such as ChatGPT, with their vast internet-scale training data, offer transformative potential to address this gap. In this study, we used both qualitative and quantitative methods to investigate how well ChatGPT can address visualization design questions. First, we quantitatively compared the ChatGPT-generated responses with anonymous online Human replies to data visualization questions on the VisGuides user forum. Next, we conducted a qualitative user study examining the reactions and attitudes of practitioners toward ChatGPT as a visualization design assistant. Participants were asked to bring their visualizations and design questions and received feedback from both Human experts and ChatGPT in randomized order. Our findings from both studies underscore ChatGPT's strengths, particularly its ability to rapidly generate diverse design options, while also highlighting areas for improvement, such as nuanced contextual understanding and fluid interaction dynamics beyond the chat interface. Drawing on these insights, we discuss design considerations for future LLM-based design feedback systems.
