🤖 AI Summary
Qualitative research faces a persistent trade-off between analytical scale and interpretive depth. Method: This paper introduces Neo-Grounded Theory—a novel methodology that integrates high-dimensional semantic embeddings (1536-D), hierarchical vector clustering, and a multi-agent collaborative coding system into the grounded theory framework, thereby unifying computational objectivity with humanistic interpretability. It supports both fully automated and human-guided human–AI collaborative analysis modes, enabling detection of latent patterns—such as identity bifurcation—that elude manual identification. Contribution/Results: Empirical evaluation demonstrates a 168× acceleration in analysis time (3 hours vs. 3 weeks), a 96% cost reduction ($50,000 → $500), intercoder reliability of 0.904 (Cohen’s κ), and generation of actionable dual-path theoretical models. This work advances qualitative research toward real-time, reproducible, and democratized paradigms.
📝 Abstract
Purpose: Neo Grounded Theory (NGT) integrates vector clustering with multi agent systems to resolve qualitative research's scale depth paradox, enabling analysis of massive datasets in hours while preserving interpretive rigor. Methods: We compared NGT against manual coding and ChatGPT-assisted analysis using 40,000 character Chinese interview transcripts. NGT employs 1536-dimensional embeddings, hierarchical clustering, and parallel agent-based coding. Two experiments tested pure automation versus human guided refinement. Findings: NGT achieved 168-fold speed improvement (3 hours vs 3 weeks), superior quality (0.904 vs 0.883), and 96% cost reduction. Human AI collaboration proved essential: automation alone produced abstract frameworks while human guidance yielded actionable dual pathway theories. The system discovered patterns invisible to manual coding, including identity bifurcation phenomena. Contributions: NGT demonstrates computational objectivity and human interpretation are complementary. Vector representations provide reproducible semantic measurement while preserving meaning's interpretive dimensions. Researchers shift from mechanical coding to theoretical guidance, with AI handling pattern recognition while humans provide creative insight. Implications: Cost reduction from $50,000 to $500 democratizes qualitative research, enabling communities to study themselves. Real-time analysis makes qualitative insights contemporaneous with events. The framework shows computational methods can strengthen rather than compromise qualitative research's humanistic commitments.
Keywords: Grounded theory; Vector embeddings; Multi agent systems; Human AI collaboration; Computational qualitative analysis