🤖 AI Summary
This study addresses a critical limitation in current learning analytics tools, wherein frequency-oriented visualizations often obscure rare yet educationally significant student feedback. To bridge the gap between quantitative visualization and qualitative educational research, the authors engaged STEM education researchers in analyzing student logs using the WordStream platform. Through an integrated approach combining thematic analysis, member checking, and mixed-methods user research, the study uncovered epistemological tensions educators face when repurposing quantitative codings for qualitative inquiry. Three core themes emerged: tool experience, disciplinary contextualization, and the integration of quantitative and qualitative paradigms. Building on these insights, the work proposes design principles for visualizations that support deep qualitative exploration, offering both theoretical grounding and practical guidance for the next generation of learning analytics tools.
📝 Abstract
Data-driven learning analytics can surface trends across a student cohort over time, helping instructors improve the learning environment. WordStream, a visualization idiom for topic evolution, has been instantiated in two platforms toward this goal: the Journal Data Dashboard, for analyzing formative assessments, and WordStream Maker, for authoring custom visualizations. Where the prior work built these platforms for education (Vis4Ed), here we examine the reverse direction (Ed4Vis): what can qualitative education research tell us about building better visualization tools? We conducted a mixed-methods expert study (n=10) in which STEM education researchers with expertise in qualitative methods and classroom assessment used both platforms to analyze student journal responses from a data visualization course. Across two cycles of thematic analysis with confirmatory checking, we report themes spanning tool experience, disciplinary context of use, and, most importantly, a core epistemological dissensus. Some instructor-researchers regarded frequency-based visualization as a productive entry point to qualitative analysis; others cautioned it can obscure rare but critical responses. We synthesize these findings into design implications for future tools that better integrate quantitative technique with qualitative inquiry. All Supplementary Materials are available at https://osf.io/z2f8d.