TaxoAlign: Scholarly Taxonomy Generation Using Language Models

2025-10-20

AI Summary

Existing automated academic survey generation methods lack systematic structural alignment between generated taxonomies and those crafted by human experts, resulting in insufficient semantic coherence and hierarchical consistency. To address this, we propose TaxoAlign—a three-stage taxonomy generation framework jointly guided by topic modeling and instruction tuning, integrating topic-aware representation learning, large language model–driven hierarchical expansion, and structure-aware refinement. To enable rigorous evaluation, we introduce CS-TaxoBench, the first high-quality, expert-annotated benchmark for computer science taxonomies, and design the first automated, quantitative evaluation framework for structural alignment and semantic coherence. Experimental results demonstrate that TaxoAlign significantly outperforms all baselines in both automated metrics and human evaluations, achieving breakthrough improvements in hierarchical plausibility, cross-level semantic consistency, and expert alignment.

Technology Category

• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures
• Machine Learning: Structured Learning
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Taxonomies play a crucial role in helping researchers structure and navigate knowledge in a hierarchical manner. They also form an important part in the creation of comprehensive literature surveys. The existing approaches to automatic survey generation do not compare the structure of the generated surveys with those written by human experts. To address this gap, we present our own method for automated taxonomy creation that can bridge the gap between human-generated and automatically-created taxonomies. For this purpose, we create the CS-TaxoBench benchmark which consists of 460 taxonomies that have been extracted from human-written survey papers. We also include an additional test set of 80 taxonomies curated from conference survey papers. We propose TaxoAlign, a three-phase topic-based instruction-guided method for scholarly taxonomy generation. Additionally, we propose a stringent automated evaluation framework that measures the structural alignment and semantic coherence of automatically generated taxonomies in comparison to those created by human experts. We evaluate our method and various baselines on CS-TaxoBench, using both automated evaluation metrics and human evaluation studies. The results show that TaxoAlign consistently surpasses the baselines on nearly all metrics. The code and data can be found at https://github.com/AvishekLahiri/TaxoAlign.

Problem

• Automating scholarly taxonomy generation using language models
• Bridging the gap between human and machine-created taxonomies
• Evaluating structural alignment and semantic coherence of taxonomies

Innovation

• Three-phase topic-based instruction-guided taxonomy generation method
• Automated evaluation framework for structural alignment and semantic coherence
• CS-TaxoBench benchmark with human-extracted taxonomies for comparison