Synthetic Contrastive Reasoning for Multi-Table Q&A

2026-06-03

AI Summary

This study addresses the challenge in multi-table question answering, where the absence of supervision signals for reasoning processes hinders effective cross-table evidence retrieval, schema linking, and compositional reasoning. To overcome this limitation, the authors introduce the first dataset of contrastive positive–negative reasoning trajectories generated by heterogeneous large language models and propose fine-tuning open-source large models using Contrastive Preference Optimization (CPO). By synthesizing high-quality preference pairs of reasoning trajectories, this approach substantially enhances the model’s multi-hop reasoning capabilities. Experimental results on Qwen3-14B, Mistral-8B, and Llama-3.1-8B demonstrate consistent improvements over conventional question-answering fine-tuning, yielding average gains of 9.7%–16.3% on the MMQA benchmark, with a maximum improvement of 21 percentage points.

Abstract

Multi-table question answering requires models to retrieve relevant evidence, link schemas, and perform compositional reasoning across relational tables. Existing multi-table Q&A resources typically provide questions and final answers but lack reasoning supervision that explains how answers are derived. To address this gap, we construct a synthetic contrastive reasoning-trace dataset for MMQA by generating validated positive traces and plausible negative traces with heterogeneous LLMs. We then use the resulting preference pairs to fine-tune open-weight LLMs with Contrastive Preference Optimization (CPO). Across Qwen3-14B, Mistral-8B, and Llama-3.1-8B, CPO achieves absolute average improvements over Q&A supervised fine-tuning ranging from 9.7%-16.3%, with gains up to 21 percentage points on MMQA. Ablations show that heterogeneous positive and negative trace generators strengthen the contrastive signal, and automated as well as human evaluations indicate that the generated pairs are largely faithful, coherent, and meaningfully contrastive.

Problem

• multi-table question answering
• reasoning supervision
• contrastive reasoning
• synthetic data
• relational tables

Innovation

• Synthetic Contrastive Reasoning
• Contrastive Preference Optimization
• Multi-Table Question Answering
• Reasoning Traces
• Heterogeneous LLMs