LH-Mix: Local Hierarchy Correlation Guided Mixup over Hierarchical Prompt Tuning

2024-12-22

arXiv.org

AI Summary

Hierarchical text classification (HTC) suffers from redundancy in global hierarchy graphs and neglects implicit semantic correlations among sibling labels. Method: We propose a text-specific, local hierarchy-aware prompt-tuning framework that (1) replaces the global hierarchy tree with instance-level local hierarchical subgraphs; (2) employs depth-wise manually designed prompt templates; and (3) introduces a novel dynamic Mixup mechanism guided by local hierarchical correlations—explicitly modeling parent-child relationships while implicitly enhancing semantic consistency among sibling labels. Contribution/Results: Our approach achieves significant improvements over state-of-the-art methods on three benchmark HTC datasets, demonstrating that local structural modeling and correlation-guided data augmentation are critical for enhancing hierarchical generalization capability.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Hierarchical text classification (HTC) aims to assign one or more labels in the hierarchy for each text. Many methods represent this structure as a global hierarchy, leading to redundant graph structures. To address this, incorporating a text-specific local hierarchy is essential. However, existing approaches often model this local hierarchy as a sequence, focusing on explicit parent-child relationships while ignoring implicit correlations among sibling/peer relationships. In this paper, we first integrate local hierarchies into a manual depth-level prompt to capture parent-child relationships. We then apply Mixup to this hierarchical prompt tuning scheme to improve the latent correlation within sibling/peer relationships. Notably, we propose a novel Mixup ratio guided by local hierarchy correlation to effectively capture intrinsic correlations. This Local Hierarchy Mixup (LH-Mix) model demonstrates remarkable performance across three widely-used datasets.

Problem

• Improve hierarchical text classification accuracy
• Capture implicit sibling/peer relationships
• Optimize local hierarchy correlation in prompts

Innovation

• Local Hierarchy Mixup (LH-Mix)
• Hierarchical Prompt Tuning
• Mixup ratio guided correlation