CARDIUM: Congenital Anomaly Recognition with Diagnostic Images and Unified Medical records

2025-10-16

AI Summary

To address the challenges of data scarcity, modality fragmentation, and low-quality inputs that limit model performance in prenatal diagnosis of congenital heart disease (CHD), this work introduces CARDIUM—the first publicly available multimodal CHD dataset—integrating fetal ultrasound images, fetal electrocardiogram (ECG) images, and structured maternal clinical records. We propose a novel multimodal Transformer architecture leveraging cross-attention mechanisms to enable end-to-end joint modeling of image and tabular features. Evaluated on CARDIUM, our model achieves an F1-score of 79.8±4.8%, outperforming unimodal image-based and tabular baselines by 11% and 50%, respectively, and significantly improving detection of rare CHD subtypes. This work establishes a high-quality benchmark dataset and a reproducible methodological framework for multimodal AI in prenatal rare-disease diagnosis.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Prenatal diagnosis of Congenital Heart Diseases (CHDs) holds great potential for Artificial Intelligence (AI)-driven solutions. However, collecting high-quality diagnostic data remains difficult due to the rarity of these conditions, resulting in imbalanced and low-quality datasets that hinder model performance. Moreover, no public efforts have been made to integrate multiple sources of information, such as imaging and clinical data, further limiting the ability of AI models to support and enhance clinical decision-making. To overcome these challenges, we introduce the Congenital Anomaly Recognition with Diagnostic Images and Unified Medical records (CARDIUM) dataset, the first publicly available multimodal dataset consolidating fetal ultrasound and echocardiographic images along with maternal clinical records for prenatal CHD detection. Furthermore, we propose a robust multimodal transformer architecture that incorporates a cross-attention mechanism to fuse feature representations from image and tabular data, improving CHD detection by 11% and 50% over image and tabular single-modality approaches, respectively, and achieving an F1 score of 79.8 $pm$ 4.8% in the CARDIUM dataset. We will publicly release our dataset and code to encourage further research on this unexplored field. Our dataset and code are available at https://github.com/BCVUniandes/Cardium, and at the project website https://bcv-uniandes.github.io/CardiumPage/

Problem

• Addresses limited public multimodal datasets for prenatal CHD detection
• Overcomes data imbalance and quality issues in congenital anomaly diagnosis
• Integrates fetal imaging with clinical records using cross-attention mechanisms

Innovation

• Multimodal dataset combining ultrasound images and clinical records
• Transformer architecture with cross-attention for data fusion
• Improved CHD detection by integrating imaging and tabular data