PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data

2026-06-04

AI Summary

This work addresses the challenge of structural missingness in multimodal medical time-series data, which manifests as either intra-modality or entire-modality missingness—two distinct patterns that existing methods struggle to handle uniformly and often decouple from downstream prediction tasks. To bridge this gap, the authors propose a prior-aware multimodal fusion framework that introduces modality- and missingness-type-specific prior initialization for the first time. By integrating flow matching, weight-shared encoders, and architectural alignment mechanisms, the model explicitly differentiates between missingness types and enables joint optimization of imputation and prediction through bidirectional information flow. Extensive experiments across multiple medical time-series benchmarks demonstrate that the proposed method consistently outperforms current baselines under diverse missingness settings, achieving state-of-the-art performance on downstream clinical prediction tasks.

Abstract

In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because electrodes detach or an entire respiratory channel is unavailable during overnight monitoring. Such missingness typically appears in two structurally distinct patterns: within-modality missing, where values are absent within an otherwise observed modality, and modality-level missing, where an entire modality is unavailable. Existing methods typically represent unobserved data implicitly through masks or missing embeddings, without learning instance-specific missing information, and most are designed for only one missingness pattern. A natural approach is to explicitly estimate the missing data; however, existing imputation methods treat missingness uniformly despite their different structural priors, and the imputation process is often isolated from downstream tasks, preventing downstream tasks from guiding imputation toward more informative representations. To address these limitations, we present PAMF, a multimodal time-series framework that explicitly handles different missingness patterns while coupling imputation with downstream prediction through prior-aware flow matching and weight sharing. Specifically, the method initializes the flow-matching source state with type-specific priors to distinguish two missing types. It further connects imputation and classification through architecturally matched encoders with weight sharing, transferring task-relevant representations into the imputation process. Experiments on multiple multimodal healthcare time-series benchmarks show that the proposed method achieves the strongest overall downstream performance across diverse datasets and missing settings compared with existing baselines.

Problem

• incomplete time series
• multimodal fusion
• missing data
• healthcare time series
• missingness patterns

Innovation

• Prior-Aware Fusion
• Multimodal Time Series
• Incomplete Data Imputation
• Flow Matching
• Weight Sharing