RadHarmony: Radiological Data Handling in the Era of Agentic AI

2026-07-24

AI Summary

This work addresses the significant standardization challenges in integrating multi-source, heterogeneous radiological imaging data—such as chest X-rays, CT, and MRI—along with their diverse annotations, including classification labels, segmentation masks, bounding boxes, and radiology reports. To this end, the authors propose RadHarmony, an open-source Python library that, for the first time, employs AI agents to guide the entire pipeline from raw data inspection to code generation and testing, while harmonizing metadata across 24 publicly available datasets. Built upon MONAI, RadHarmony provides a unified, mappable dataset interface with on-disk caching and consistent support for multimodal annotations. Leveraging a Vision Transformer architecture, it enables pretraining without dataset-specific code. The released RadHarmony-ViT benchmark model successfully unifies three heterogeneous chest X-ray datasets, with both code and models made openly available.

Abstract

Training deep learning models on radiological images requires integrating heterogeneous datasets across different sources, file formats, directory layouts, label schemas, and annotation types. We present RadHarmony, an open-source Python library that provides a unified API for loading, harmonizing, and augmenting radiological datasets, with a primary focus on chest radiographs and early support for computed tomography (CT) and magnetic resonance imaging (MRI). RadHarmony standardizes metadata from 24 public datasets into a single tabular format, wraps MONAI's map-style datasets for deep-learning-ready sample delivery with optional on-disk caching, and supports classification labels, segmentation masks, bounding boxes, and radiology report text through a single interface, with an interactive visualization tool for dataset exploration and verification. To lower the barrier for integrating new datasets, RadHarmony introduces an AI-agent skill that guides the full integration workflow from raw data inspection through code generation and testing. We demonstrate the library's utility by pretraining RadHarmony-ViT, a reference vision transformer baseline that combines three heterogeneous chest radiograph datasets with no dataset-specific code. The code and pretrained model weights are available at https://github.com/f10409/RadHarmony.

Problem

• radiological data harmonization
• heterogeneous datasets
• deep learning
• medical imaging
• data integration

Innovation

• RadHarmony
• Agentic AI
• Radiological Data Harmonization
• Unified Medical Imaging API
• Vision Transformer Pretraining