Learning Through Game: Skewed Transfer of Tabular Knowledge to Strengthen Image Model

2026-09-26

AI Summary

This study addresses the challenges of missing tabular data at test time and modality imbalance in multimodal learning, which constrain the performance of image-based models. To this end, we propose a skewed knowledge transfer method that injects tabular knowledge into the image model via a multimodal shared head and an asymmetric transfer mechanism. Furthermore, a gradient fusion strategy based on two-step Nash bargaining is designed to adaptively integrate cross-modal gradients, thereby overcoming modality discrepancies. We provide theoretical guarantees for both the Pareto improvement and convergence of the proposed approach. Extensive experiments demonstrate that our method significantly enhances unimodal inference performance using only images across multiple datasets.

Abstract

Multimodal tabular-image learning is gaining growing attention, yet it faces challenges due to tabular data unavailable at test time. A practical solution involves transferring tabular knowledge to images during training to enhance the performance of image models at inference. However, the overlooked yet important challenges lie in the modality imbalance between images and tables, as well as their asymmetric modality relationship in cross-modal transfer, which limits the auxiliary role of tabular data. To address these issues, we propose Skewed Knowledge Transfer (SKT), which asymmetrically transfers tabular knowledge to improve the image model by adaptive integration of modality gradients in a shared parameter space. Specifically, we first introduce a multimodal shared head, which allows the model to benefit from cross-modal structure without adding additional parameters. We then design a two-step Nash Bargaining strategy to effectively leverage tabular gradients. In the first step, SKT seeks a point of modality balance and uses preference awareness in the second step to steer combined gradients toward image-beneficial directions. Furthermore, we theoretically analyze the Pareto improvement and convergence of SKT. To this end, tabular knowledge is explicitly transferred to enhance image models. Empirical experiments on widely used tabular-image datasets reveal that SKT consistently improves image unimodal performance by using tabular data as auxiliary information.

Problem

• multimodal tabular-image learning
• knowledge transfer
• modality imbalance
• asymmetric modality relationship

Innovation

• Skewed Knowledge Transfer
• Nash Bargaining
• Multimodal Shared Head
• Tabular-Image Learning
• Pareto Improvement