TabRank: Chain-of-Thought Distillation for Table Re-Rankers

2026-07-27

AI Summary

This work addresses the limited semantic understanding and reasoning capabilities of existing re-ranking methods in table retrieval, particularly their poor generalization in multi-table and cross-domain scenarios. To overcome this, the authors propose TabRank, a novel framework that introduces explicit chain-of-thought (CoT) distillation into table re-ranking for the first time. Leveraging 6,728 reasoning trajectories generated by large language models, TabRank trains a lightweight neural re-ranker to enhance its ability to reason over structured tabular data. Experimental results demonstrate substantial performance gains across multiple table-based question answering benchmarks, with Acc@10 improvements of 30.5% on HybridQA, 15.2% on SQA, 52.9% on TabFact, and 13.1% on TATQA, confirming the effectiveness and strong generalization of the proposed approach.

Abstract

The ability to retrieve relevant tables for answering questions is a key task for structured information retrieval. Multi-stage retrieval systems rely heavily on rerankers to refine candidate lists produced by efficient first-stage retrievers. As a result, neural rerankers and LLM-based reranking methods have become increasingly important due to their superior capacity for semantic understanding and reasoning compared to conventional sparse or dense retrieval models. Recently, Large Reasoning Models (LRMs) equipped with explicit chain-of-thought (CoT) reasoning have shown strong improvements in ranking quality in unstructured passage retrieval. In this work, we present TabRank, a framework for training reasoning rerankers for Tabular Retrieval. We first present a comprehensive dataset of 6728 reasoning traces for tabular reranking on the Natural Questions Tables dataset. We then explore two variants of training a compact reasoning model on these reasoning traces: explicit CoT distillation and conditioning the student reranker on the teacher's reasoning trace within the prompt. We stress-test TabRank on several out-of-distribution generalization settings on diverse domains and multi-table scenarios. Our approach significantly improves performance across a variety of table retrieval datasets, increasing Acc@10 by 30.5% on HybridQA, 15.2% on SQA, 52.9% on TabFact, and 13.1% on TATQA subsets of the Multi-Table QA Benchmark compared to the base model. Notably, TabRank generalizes effectively to multi-table reasoning. Our code, data and models are available at https://github.com/AdarshSingh7647/TabRanker

Problem

• table retrieval
• reranking
• chain-of-thought
• reasoning
• out-of-distribution generalization

Innovation

• Chain-of-Thought Distillation
• Table Re-Ranking
• Large Reasoning Models
• Out-of-Distribution Generalization
• Multi-Table QA