🤖 AI Summary
This study addresses the lack of auditability in large language models (LLMs) when applied to qualitative data analysis, a limitation stemming from their opaque processing. To remedy this, the authors propose QualAnalyzer, an open-source Chrome extension that atomizes each data segment and independently logs prompts, inputs, and outputs, thereby enabling full traceability and auditability of the LLM analytical process for the first time. Integrating browser-based functionality, Google Workspace compatibility, and systematic prompt engineering, QualAnalyzer facilitates rigorous comparative analysis between LLM-generated and human judgments. Empirical validation through two case studies—essay scoring and interview thematic coding—demonstrates that this approach substantially enhances transparency and methodological robustness in LLM-assisted qualitative research.
📝 Abstract
Large language models are increasingly used for qualitative data analysis, but many workflows obscure how analytic conclusions are produced. We present QualAnalyzer, an open-source Chrome extension for Google Workspace that supports atomistic LLM analysis by processing each data segment independently and preserving the prompt, input, and output for every unit. Through two case studies -- holistic essay scoring and deductive thematic coding of interview transcripts -- we show that this approach creates a legible audit trail and helps researchers investigate systematic differences between LLM and human judgments. We argue that process auditability is essential for making LLM-assisted qualitative research more transparent and methodologically robust.