Marginal Response Surface Elicitation for Zero-Label Tabular Learning

2026-09-30

AI Summary

This study addresses the challenge that tabular data learning heavily relies on labeled samples, making zero-label prediction difficult. To this end, it proposes MARS, a framework that leverages large language models (LLMs) and feature-level prompt engineering to extract prior knowledge. Specifically, MARS constructs reusable feature functions through median aggregation and generates a zero-shot classifier via a weighted summation mechanism, enabling inference without additional LLM queries. This approach efficiently transforms LLM priors into a stable classifier, significantly reducing end-to-end inference costs. Evaluated across eight benchmark tasks, MARS achieves state-of-the-art average AUC and AP scores, outperforming direct prompting by 1.97% and 6.21%, respectively.

Abstract

Tabular learning uses structured data to predict target outcomes. Traditionally, this process has relied on labeled data. However, large language models (LLMs) can be used to elicit domain priors based on the task description and feature semantics, thereby enabling predictions without labeled data. We propose Marginal Response Surface Elicitation (MARS), a method that transforms feature-level LLM priors into a reusable, zero-shot tabular classifier. To construct this classifier, MARS selects representative values for each feature from unlabeled data and prompts the LLM to provide corresponding class support scores and feature weights. It then aggregates multiple responses using the median to construct feature response functions, and makes predictions through their weighted sum without further LLM queries. Across eight tabular benchmark tasks, MARS achieves the highest average AUC and AP, outperforming direct prompting by 1.97 and 6.21 percentage points respectively, while substantially reducing end-to-end costs. Evaluations with LLMs of different sizes further demonstrate its predictive advantage over direct prompting.

Problem

• Zero-label tabular learning
• Large language models
• Domain priors
• Zero-shot classification

Innovation

• Zero-Label Tabular Learning
• Marginal Response Surface Elicitation
• Large Language Models
• Zero-Shot Classification
• Domain Prior Elicitation