From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs

2026-01-20

AI Summary

This work proposes an automated axial coding approach that integrates large language models (LLMs) with clustering algorithms to efficiently transform raw statements from political debates into structured, high-level conceptual categories, thereby supporting interpretable qualitative analysis. The method strategically combines embedding-based clustering with direct LLM-driven grouping to balance coverage and semantic alignment. Experiments on Dutch parliamentary debate data demonstrate that the clustering-based approach yields optimal coverage and clear structural organization, while LLM-based grouping produces more concise categories with stronger semantic consistency. To the best of our knowledge, this study presents the first fully automated axial coding pipeline and releases a comprehensive annotated dataset, establishing a novel paradigm for hierarchical semantic modeling of political discourse.

Technology Category

• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Machine Learning: Clustering
• Knowledge Representation and Reasoning: Preferences

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Axial coding is a commonly used qualitative analysis method that enhances document understanding by organizing sentence-level open codes into broader categories. In this paper, we operationalize axial coding with large language models (LLMs). Extending an ensemble-based open coding approach with an LLM moderator, we add an axial coding step that groups open codes into higher-order categories, transforming raw debate transcripts into concise, hierarchical representations. We compare two strategies: (i) clustering embeddings of code-utterance pairs using density-based and partitioning algorithms followed by LLM labeling, and (ii) direct LLM-based grouping of codes and utterances into categories. We apply our method to Dutch parliamentary debates, converting lengthy transcripts into compact, hierarchically structured codes and categories. We evaluate our method using extrinsic metrics aligned with human-assigned topic labels (ROUGE-L, cosine, BERTScore), and intrinsic metrics describing code groups (coverage, brevity, coherence, novelty, JSD divergence). Our results reveal a trade-off: density-based clustering achieves high coverage and strong cluster alignment, while direct LLM grouping results in higher fine-grained alignment, but lower coverage 20%. Overall, clustering maximizes coverage and structural separation, whereas LLM grouping produces more concise, interpretable, and semantically aligned categories. To support future research, we publicly release the full dataset of utterances and codes, enabling reproducibility and comparative studies.

Problem

• axial coding
• political debates
• large language models
• qualitative analysis
• hierarchical representation

Innovation

• axial coding
• large language models
• ensemble LMs
• hierarchical text representation
• political debate analysis