Human Preference aligned Tabular Similarity

2026-07-27

AI Summary

This work addresses a critical gap in existing task-agnostic tabular embedding methods, which, despite strong performance on predictive tasks, fail to produce similarity rankings aligned with human preferences and lack reliable evaluation mechanisms. To bridge this gap, the study introduces human preference alignment into tabular embedding evaluation for the first time, proposing a novel assessment framework tailored to real-world business scenarios such as product lifecycle management. By integrating task-agnostic embeddings with human preference data, the framework overcomes the limitations of conventional downstream metrics in evaluating embedding trustworthiness. Empirical validation through a product lifecycle management use case not only exposes the shortcomings of current methods in preference alignment but also demonstrates the effectiveness and practicality of the proposed evaluation pipeline.

Abstract

Task-agnostic tabular embeddings are increasingly used for similarity search in real-world business systems such as Product Lifecycle Management (PLM). However, leading embedding approaches are optimized primarily for prediction tasks - not for producing human preference aligned similarity rankings. We argue that standard downstream metrics are insufficient to fully assess embedding trustworthiness for similarity search and that human preference aligned evaluation is a necessary and currently missing component. We present a concrete evaluation procedure and illustrate the problem through a PLM use case.

Problem

• tabular embeddings
• similarity search
• human preference alignment
• embedding evaluation
• trustworthiness

Innovation

• human preference alignment
• tabular embeddings
• similarity search
• evaluation methodology
• task-agnostic representation