🤖 AI Summary
Existing approaches to modeling medical tabular data often neglect column-level contextual information (e.g., header semantics), require labor-intensive, manual preprocessing, and lack semantic interpretability. Method: We propose TabText—a novel framework that systematically encodes tabular structure into promptable natural language text, enabling end-to-end semantic representation learning via large language models (LLMs). TabText integrates structure-aware prompt engineering with a multi-task health prediction fine-tuning paradigm, supporting zero- or light-preprocessing modeling while remaining compatible with conventional feature fusion. Results: Evaluated on nine clinical prediction tasks, TabText alone establishes high-performance, lightweight baselines. When fused with traditional features, it yields an average AUC improvement of 6%, with even the worst-case gain reaching 6%, significantly enhancing model robustness and generalization across diverse healthcare scenarios.
📝 Abstract
Tabular data is essential for applying machine learning tasks across various industries. However, traditional data processing methods do not fully utilize all the information available in the tables, ignoring important contextual information such as column header descriptions. In addition, pre-processing data into a tabular format can remain a labor-intensive bottleneck in model development. This work introduces TabText, a processing and feature extraction framework that extracts contextual information from tabular data structures. TabText addresses processing difficulties by converting the content into language and utilizing pre-trained large language models (LLMs). We evaluate our framework on nine healthcare prediction tasks ranging from patient discharge, ICU admission, and mortality. We show that 1) applying our TabText framework enables the generation of high-performing and simple machine learning baseline models with minimal data pre-processing, and 2) augmenting pre-processed tabular data with TabText representations improves the average and worst-case AUC performance of standard machine learning models by as much as 6%.
