Problem
Research questions and friction points this paper is trying to address.
Medical Time Series
Morphology
Clinical Classification
Vision-Language Models
Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective
🤖 AI Summary
Existing medical time series methods predominantly process numerical features while overlooking waveform morphology, which encodes rich pathological information. To address this limitation, this work proposes ViRe, a framework that introduces the first cross-modal retrieval mechanism grounded in visual morphological priors. Specifically, ViRe leverages a frozen pre-trained vision-language model to extract waveform morphology as query vectors, subsequently retrieving relevant evidence from raw numerical features via an attention mechanism. This design enables effective fusion of visual and numerical modalities to enhance classification performance. Extensive experiments demonstrate that ViRe consistently outperforms ten baseline methods across six benchmark datasets, achieving a 6.42% relative improvement over the current state-of-the-art model.
📝 Abstract
Medical time series (MedTS) underpin many clinical classification tasks, yet existing methods usually represent them only as numerical sequences and underuse the morphology that is explicit in waveform inspection. To bridge this gap, we introduce Vision-Informed Retrieval (ViRe), which uses a frozen VLM-derived waveform representation as a morphology-aware Query to guide retrieval from raw numerical MedTS features. Specifically, a Vision Query is extracted using pre-trained vision-language models (VLMs) to obtain morphology-aware priors from waveform plots. A tailored attention-based cross-modal retrieval mechanism then uses the Vision Query to select morphology-relevant temporal and channel evidence from the numerical representation. ViRe demonstrates strong effectiveness against ten established baselines, yielding an overall 6.42% relative improvement over the previous state of the art across six public benchmarks. Code, training scripts, and reproducibility materials are publicly available in the GitHub Repository: https://github.com/Levi-Ackman/ViRe.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Vision-Informed Retrieval
Medical Time Series
Vision-Language Models
Cross-modal Attention
Morphology-aware Representation
🔎 Similar Papers
No similar papers found.
