AutoIntent: AutoML for Text Classification

2025-09-25

AI Summary

Current AutoML tools lack end-to-end automation for text classification, particularly in embedding model selection, classifier hyperparameter optimization, decision threshold tuning, multi-label classification, and out-of-distribution (OOD) detection. To address these gaps, we propose a modular, scikit-learn–compatible automated text classification framework. Our method unifies embedding selection, classifier optimization, and dynamic threshold tuning into a single end-to-end pipeline—the first to natively support both multi-label classification and intent-out-of-scope (OOD) detection. The framework enables configurable accuracy-efficiency trade-offs, enhancing adaptability and deployment flexibility. Extensive experiments on standard intent classification benchmarks demonstrate that our approach consistently outperforms state-of-the-art AutoML tools in accuracy, macro-F1 score, and OOD detection performance. The design ensures practical usability, scalability, and seamless integration into existing ML workflows.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Natural Language Processing: Text Classification & Sentiment Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

AutoIntent is an automated machine learning tool for text classification tasks. Unlike existing solutions, AutoIntent offers end-to-end automation with embedding model selection, classifier optimization, and decision threshold tuning, all within a modular, sklearn-like interface. The framework is designed to support multi-label classification and out-of-scope detection. AutoIntent demonstrates superior performance compared to existing AutoML tools on standard intent classification datasets and enables users to balance effectiveness and resource consumption.

Problem

• Automating end-to-end text classification with model selection and optimization
• Supporting multi-label classification and out-of-scope detection capabilities
• Balancing classification effectiveness with computational resource consumption

Innovation

• Automated embedding selection and classifier optimization
• Modular sklearn-like interface for end-to-end automation
• Multi-label classification with out-of-scope detection capability