Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classification

2026-06-03

AI Summary

This work addresses the challenge of multimodal classification with highly heterogeneous image and tabular data under arbitrary modality missingness, where existing methods struggle due to coarse-grained alignment that fails to capture fine-grained semantic discrepancies and distributional mismatches. To overcome this limitation, we propose DFPL, a novel framework that jointly achieves fine-grained distribution matching and semantic alignment in a prototype space. DFPL introduces Shared-Specific Prototype Modeling (SSPM) to disentangle modality-invariant and modality-specific features while suppressing intra-modality redundancy, employs Prototype-guided Fine-grained Alignment (PFA) to enhance cross-modal semantic consistency, and designs Category-aware Multi-scale Aggregation (CMA) to adaptively fuse global and prototype-level information. Extensive experiments across three image-tabular benchmark datasets under diverse modality-missing scenarios demonstrate that DFPL consistently outperforms state-of-the-art methods, confirming its effectiveness and robustness.

Abstract

The missing-modality problem poses a significant challenge in image-tabular multimodal learning across a wide range of multimedia applications, including product understanding, recommendation systems, and medical diagnosis. This challenge is particularly pronounced when the two modalities are highly heterogeneous, as images and tabular attributes differ substantially in their semantic granularity and data distributions. Existing methods learn modality-invariant representations through disentanglement and alignment over global token-averaged features, capturing only coarse cross-modal consistency and overlooking fine-grained semantic and distributional misalignment, which hampers the exploitation of complementary cues under missing modalities. To address this, we propose DFPL, a novel framework for fine-grained prototype learning. Specifically, Shared-Specific Prototype Modeling (SSPM) extracts compact and diverse shared and modality-specific prototypes, and further performs prototype-level disentanglement to suppress redundant intra-modality correlations. Additionally, we propose a Prototype-guided Fine-grained Alignment (PFA) module that jointly enforces prototype-level distribution matching and prototype-to-class semantic alignment within a unified prototype space, thereby preserving both fine-grained distributional and semantic consistency across modalities. We further introduce a Class-aware Multi-scale Aggregation (CMA) module to adaptively aggregate shared semantics and modality-specific characteristics from global and prototype levels for robust predictions. Extensive experiments on three diverse image-tabular benchmarks demonstrate the superiority of our method compared to the previous approaches under various missing-modality settings. Code will be made publicly available.

Problem

• missing-modality
• image-tabular classification
• fine-grained alignment
• multimodal learning
• heterogeneous data

Innovation

• fine-grained prototype learning
• modality disentanglement
• prototype-guided alignment
• missing-modality robustness
• image-tabular fusion