Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation

2026-02-17

AI Summary

This work proposes MARL-Rad, a novel multi-agent reinforcement learning framework for radiology report generation that addresses the limitations of existing approaches relying on fixed large language models manually orchestrated into agent pipelines without role-specific optimization for radiological tasks. MARL-Rad decomposes chest X-ray interpretation into multiple region-specific agents and a global integrator agent, enabling, for the first time, end-to-end joint policy training within a multi-agent system for radiology reporting. The framework integrates medical images and text, employing a reward function grounded in clinically verifiable metrics to jointly optimize report accuracy, detail richness, and left-right consistency. Evaluated on MIMIC-CXR and IU X-ray datasets, MARL-Rad achieves state-of-the-art performance across clinical efficacy metrics—including RadGraph, CheXbert, and GREEN—and blind evaluations indicate that its generated reports are comparable in quality to authentic clinical reports.

Abstract

We propose MARL-Rad, a multi-modal multi-agent reinforcement learning framework for radiology report generation that trains the entire agentic system on policy within its deployed radiology workflow. MARL-Rad addresses the limitation of post-hoc agentization, where fixed LLMs are organized into hand-designed agentic workflows without being optimized for their assigned roles. Our framework decomposes chest X-ray interpretation into region-specific agents and a global integrating agent, and jointly optimizes them using clinically verifiable rewards. Experiments on the MIMIC-CXR and IU X-ray datasets show that MARL-Rad consistently improves clinical efficacy metrics such as RadGraph, CheXbert, and GREEN scores, achieving state-of-the-art clinical efficacy performance. Further analyses show that MARL-Rad improves laterality consistency and produces more accurate and detailed reports. A blinded clinician evaluation further suggests that MARL-Rad produces reports clinically comparable to ground-truth reports.

Problem

• radiology report generation
• multi-agent reinforcement learning
• clinical efficacy
• multi-modal learning
• LLM agentization

Innovation

• multi-agent reinforcement learning
• radiology report generation
• clinical reward optimization
• multi-modal AI
• end-to-end agentic training