LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data

2025-08-12

AI Summary

This paper addresses the challenge of modeling tabular data in zero-shot and few-shot learning scenarios. We propose a training-free prototypical estimation framework leveraging large language models (LLMs), which eliminates reliance on in-context examples. Our method introduces a novel “sample-free prompting mechanism” that directly generates feature-level prototypes from task and feature descriptions alone. These prototypes are then refined via prototype enhancement using only a small number of labeled samples—without fine-tuning the LLM or training any downstream classifier. The core innovation lies in decoupling prototype construction from instance-based demonstrations, enabling description-driven, scalable, and robust zero/few-shot tabular learning. Extensive experiments on multiple benchmark datasets demonstrate substantial improvements in classification accuracy, validating the framework’s strong generalization capability and practical deployability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent breakthroughs in large language models (LLMs) have opened the door to in-depth investigation of their potential in tabular data modeling. However, effectively utilizing advanced LLMs in few-shot and even zero-shot scenarios is still challenging. To this end, we propose a novel LLM-based prototype estimation framework for tabular learning. Our key idea is to query the LLM to generate feature values based example-free prompt, which solely relies on task and feature descriptions. With the feature values generated by LLM, we can build a zero-shot prototype in a training-free manner, which can be further enhanced by fusing few-shot samples, avoiding training a classifier or finetuning the LLMs. Thanks to the example-free prompt and prototype estimation, ours bypasses the constraints brought by the example-based prompt, providing a scalable and robust framework. Extensive experiments demonstrate the effectiveness of ours in zero and few-shot tabular learning.

Problem

• Utilizing LLMs for zero-shot tabular data modeling
• Generating feature values without example-based prompts
• Enhancing prototypes by fusing few-shot samples

Innovation

• LLM-based prototype estimation framework
• Example-free prompt for feature generation
• Training-free zero-shot prototype construction