A multi-agent system for spine MRI report generation from multi-sequence imaging

2026-06-07

AI Summary

This work addresses the challenge of generating accurate and interpretable spinal MRI reports by effectively integrating multi-sequence imaging data—a task where existing methods fall short. The authors propose SpineAgent, a multi-agent system that leverages large-scale clinical data to construct patient-level embeddings using a DINOv3 encoder and a multi-sequence continuous training synthesizer. Thirty-seven specialized agents collaboratively perform lesion localization, cross-sequence feature fusion, and structured report generation. The framework enables bidirectional image–report retrieval and produces explainable outputs. Evaluated across multi-vendor and multi-cohort settings, SpineAgent achieves state-of-the-art performance both in automated metrics and in assessments by five radiologists, marking a significant advance in multimodal MRI analysis and the automation of clinical reporting.

Abstract

Spinal pathology is a leading cause of pain and disability worldwide. Spine MRI is central to clinical evaluation, yet its interpretation remains complex and time-consuming, requiring integration of information across multiple imaging sequences and anatomical regions. Despite recent advances in automated MRI analysis, effectively combining multi-sequence data while preserving sequence-specific diagnostic information remains an open challenge. Here we present SpineAgent, a multi-agent framework for spine MRI report generation built upon a multi-sequence foundation model trained on routine clinical data from 32,047 patients and 453,683 MRI series, comprising a total of 13,441,191 MRI slices. To accommodate diverse modalities of sequences, we first pre-train two DINOv3-based encoders separately on T1- and T2-weighted sequences. We then introduce a continual training strategy that learns a synthesizer to embed images of other sequences using the T1 and T2 encoders, producing patient-level embedding that integrates various signals across MRI sequences. Using these embeddings, SpineAgent achieves state-of-the-art performance, and demonstrates strong generalizability under cross-manufacturer and cross-cohort evaluation. Beyond classification, SpineAgent enables pathology localization by identifying findings-relevant slices and segmenting pathological regions. It also supports multimodal image-report retrieval, providing a solid foundation for scalable and explainable MRI report generation. We further integrate these validated capabilities of SpineAgent into 37 specialized agents. Finally, we incorporate their outputs as structured tokens within a Medical Report Agent trained end-to-end for report generation. Through both automated metrics and expert evaluation by five radiologists, SpineAgent achieves leading performance in spine MRI report generation.

Problem

• spine MRI
• multi-sequence imaging
• report generation
• medical image analysis
• diagnostic integration

Innovation

• multi-agent system
• multi-sequence MRI
• foundation model
• continual training
• pathology localization