🤖 AI Summary
Traditional unsupervised text analysis suffers from poor interpretability and weak semantic coherence in data-scarce domains. To address this, we propose Recursive Topic Partitioning (RTP), the first framework that deeply integrates problem-driven binary semantic tree construction with large language models (LLMs), explicitly encoding clustering logic and transforming analytical pathways into structured prompts—thereby enabling interpretable clustering and controllable generation in a closed loop. RTP synergizes recursive semantic segmentation, contextual reasoning, and prompt engineering to jointly support topic discovery and data synthesis. Experiments demonstrate that RTP-generated semantic trees significantly outperform keyword-based methods (e.g., BERTopic) in structural quality and coherence; it achieves substantial gains in few-shot classification tasks; and it enables precise, semantics-guided controllable text generation aligned with user-specified semantic features.
📝 Abstract
Unsupervised analysis of text corpora is challenging, especially in data-scarce domains where traditional topic models struggle. While these models offer a solution, they typically describe clusters with lists of keywords that require significant manual effort to interpret and often lack semantic coherence. To address this critical interpretability gap, we introduce Recursive Thematic Partitioning (RTP), a novel framework that leverages Large Language Models (LLMs) to interactively build a binary tree. Each node in the tree is a natural language question that semantically partitions the data, resulting in a fully interpretable taxonomy where the logic of each cluster is explicit. Our experiments demonstrate that RTP's question-driven hierarchy is more interpretable than the keyword-based topics from a strong baseline like BERTopic. Furthermore, we establish the quantitative utility of these clusters by showing they serve as powerful features in downstream classification tasks, particularly when the data's underlying themes correlate with the task labels. RTP introduces a new paradigm for data exploration, shifting the focus from statistical pattern discovery to knowledge-driven thematic analysis. Furthermore, we demonstrate that the thematic paths from the RTP tree can serve as structured, controllable prompts for generative models. This transforms our analytical framework into a powerful tool for synthesis, enabling the consistent imitation of specific characteristics discovered in the source corpus.