Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification

2026-01-23

AI Summary

This work addresses the high inference cost and latency that hinder the deployment of large language models (LLMs) in zero-shot text classification. To overcome this bottleneck, the authors propose a closed-loop agent framework that leverages the LLM as an intelligent data curator. Within this framework, the LLM dynamically generates training examples, performs error analysis on a lightweight classifier, and synthesizes targeted samples to iteratively refine the training data. Evaluated on four mainstream benchmarks, the approach significantly outperforms standard zero-shot and few-shot baselines, achieving high accuracy while substantially improving inference efficiency. This enables practical, deployable zero-shot classification without sacrificing performance.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) and high-capacity encoders have advanced zero and few-shot classification, but their inference cost and latency limit practical deployment. We propose training lightweight text classifiers using dynamically generated supervision from an LLM. Our method employs an iterative, agentic loop in which the LLM curates training data, analyzes model successes and failures, and synthesizes targeted examples to address observed errors. This closed-loop generation and evaluation process progressively improves data quality and adapts it to the downstream classifier and task. Across four widely used benchmarks, our approach consistently outperforms standard zero and few-shot baselines. These results indicate that LLMs can serve effectively as data curators, enabling accurate and efficient classification without the operational cost of large-model deployment.

Problem

• zero-shot classification
• large language models
• inference cost
• latency
• lightweight classifiers

Innovation

• closed-loop agentic framework
• zero-shot classification
• LLM-as-curator
• dynamic data synthesis
• lightweight classifier