TEmBed-T: A Multi-Dimensional Benchmark for Table-Level Embeddings

2026-07-27

AI Summary

This work addresses the lack of systematic evaluation of table-level embeddings in multi-task settings, which hinders a comprehensive assessment of their downstream effectiveness. To overcome the limitations of single-task benchmarks focused solely on retrieval, we introduce TEmBed-T—a multidimensional benchmark encompassing tasks such as retrieval, classification, and data lake discovery. TEmBed-T integrates diverse table embedding models and establishes a unified evaluation protocol with consistent metrics across tasks. Empirical results demonstrate that no single embedding model consistently outperforms others across all tasks, revealing that the quality of table-level embeddings must be evaluated holistically through multiple dimensions rather than relying exclusively on retrieval performance.

Abstract

Tabular data is the dominant structured-data modality, and learning table representations has become a core research direction. Table-level embeddings in particular underpin a wide range of applications, including table retrieval, data lake discovery, and table classification. Despite their importance, there is still limited understanding of how different embedding approaches behave across tasks, making systematic evaluation and analysis essential. In this work, we introduce a systematic evaluation of table-level embeddings that captures several complementary properties required for downstream effectiveness. We realize this evaluation by extending TEmBed, a recently proposed testbed for tabular embeddings, whose table-level coverage is currently limited to a single retrieval task. An empirical study over the TEmBed model pool confirms that no single model excels across all tasks, demonstrating that table-level embedding quality cannot be reduced to retrieval alone.

Problem

• table-level embeddings
• tabular data
• benchmark
• systematic evaluation
• downstream tasks

Innovation

• table-level embeddings
• multi-dimensional benchmark
• systematic evaluation
• tabular data representation
• TEmBed-T