TextSLIP: Text Self-Supervised CLIP for Medical Report Generation

2026-07-24

AI Summary

This work addresses the limitation of existing CLIP-based approaches in medical report generation, which struggle to support complex descriptions due to insufficient fine-grained semantic supervision on text embeddings. To overcome this, the authors propose TextSLIP, a novel framework that extends CLIP by introducing, for the first time, self-supervised contrastive learning within the text modality to enhance the discriminability of textual representations. This refinement provides the visual encoder with more precise linguistic supervision signals. The model is pretrained on large-scale brain MRI–text pairs and subsequently fine-tuned on downstream radiology report generation tasks. Experiments on a dataset of 7 million brain MRI scans demonstrate that TextSLIP significantly outperforms standard CLIP baselines, and ablation studies confirm that the proposed text self-supervision mechanism effectively improves cross-modal alignment and generation performance.

Abstract

Automating radiology report generation is important for improving reporting consistency and clinical workflows . While Contrastive Language--Image Pretraining (CLIP) has advanced medical vision language modeling, existing CLIP-style approaches may still provide insufficient fine-grained semantic supervision for complex report generation. Standard CLIP primarily optimizes cross-modal alignment, without explicitly structuring the textual embedding space that guides visual representation learning. To address this limitation, we propose TextSLIP, a general medical vision-language pretraining framework that augments CLIP with intra-modal text contrastive learning. By improving textual embedding discriminability through self-supervised augmented text pairs, TextSLIP is designed to provide finer-grained linguistic supervision to the visual encoder. As an initial validation, we pretrain TextSLIP on a curated dataset of 7 million brain MRI image-text pairs and fine-tune the pretrained visual encoder within a report generation architecture. In controlled comparisons with CLIP-style baselines, TextSLIP shows consistent improvements on report generation metrics. Ablation studies further suggest that text-side self-supervision contributes to the observed gains. These results indicate that text-level contrastive learning is a promising direction for improving medical visual-textual alignment, while broader validation across additional medical domains remains an important next step.

Problem

• medical report generation
• CLIP
• fine-grained semantic supervision
• visual-textual alignment
• text embedding

Innovation

• TextSLIP
• intra-modal text contrastive learning
• self-supervised learning
• medical vision-language pretraining
• fine-grained semantic supervision