Robust LLM-based Column Type Annotation via Prompt Augmentation with LoRA Tuning

2025-12-27

AI Summary

Existing column type annotation (CTA) methods face two key bottlenecks: encoder-based models suffer from poor cross-domain generalization and require costly retraining; while large language models (LLMs) applied via multiple-choice prompting exhibit high template sensitivity and suboptimal F1 scores. To address these issues, we propose Prompt-LoRA—a novel framework that synergistically integrates multi-template prompt augmentation with efficient LoRA-based parameter tuning, without increasing inference overhead. By constructing diverse prompt templates, our approach enhances data robustness and mitigates prompt sensitivity. Extensive experiments on multiple CTA benchmarks demonstrate that Prompt-LoRA achieves significantly higher weighted F1 than single-template fine-tuning baselines. Moreover, it maintains strong and stable performance across cross-dataset evaluations and varying prompt structures, effectively balancing generalization capability and deployment efficiency.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Column Type Annotation (CTA) is a fundamental step towards enabling schema alignment and semantic understanding of tabular data. Existing encoder-only language models achieve high accuracy when fine-tuned on labeled columns, but their applicability is limited to in-domain settings, as distribution shifts in tables or label spaces require costly re-training from scratch. Recent work has explored prompting generative large language models (LLMs) by framing CTA as a multiple-choice task, but these approaches face two key challenges: (1) model performance is highly sensitive to subtle changes in prompt wording and structure, and (2) annotation F1 scores remain modest. A natural extension is to fine-tune large language models. However, fully fine-tuning these models incurs prohibitive computational costs due to their scale, and the sensitivity to prompts is not eliminated. In this paper, we present a parameter-efficient framework for CTA that trains models over prompt-augmented data via Low-Rank Adaptation (LoRA). Our approach mitigates sensitivity to prompt variations while drastically reducing the number of necessary trainable parameters, achieving robust performance across datasets and templates. Experimental results on recent benchmarks demonstrate that models fine-tuned with our prompt augmentation strategy maintain stable performance across diverse prompt patterns during inference and yield higher weighted F1 scores than those fine-tuned on a single prompt template. These results highlight the effectiveness of parameter-efficient training and augmentation strategies in developing practical and adaptable CTA systems.

Problem

• Improves column type annotation robustness across diverse prompts
• Reduces computational cost of fine-tuning large language models
• Enhances performance stability and accuracy in schema alignment

Innovation

• LoRA tuning for parameter-efficient fine-tuning
• Prompt augmentation to reduce sensitivity variations
• Training on diverse templates for robust performance