A unified multi-task framework enables interpretable chest radiograph analysis

2026-06-02

AI Summary

Current AI systems for medical imaging often operate as black boxes, lacking the transparency and multitask coordination required for trustworthy clinical decision-making. This work proposes IMT-CXR, a novel framework that uniquely integrates explainability with multitask learning to emulate radiologists’ diagnostic workflows. IMT-CXR simultaneously performs disease classification, lesion localization, anatomical segmentation, and radiology report generation within a unified Transformer architecture. By incorporating medical instruction fine-tuning, the model establishes an interpretable, traceable pathway from anatomical findings to diagnostic conclusions. Evaluated across ten chest X-ray benchmarks, the framework demonstrates strong performance, with 66% of its generated reports rated by radiologists in blind assessments as superior or equivalent to original clinical reports—significantly enhancing the clinical credibility and practical utility of AI-assisted diagnosis.

Abstract

While multimodal deep learning has advanced medical imaging analysis, existing black-box systems \textcolor{black}{may remain confined to isolated tasks, often overlooking} the trust-sensitive nature of clinical diagnosis as a multi-task process. We propose IMT-CXR (Interpretable Multi-task Transformer for Chest X-ray Analysis), a framework that emulates radiologists' diagnostic workflow through three evidence-driven stages: 1) Disease recognition; 2) Attribute characterization (e.g., size, location, severity quantification); 3) Evidence-integrated report generation with traceable decision pathways. The framework employs a unified transformer architecture optimized via medical-domain instruction tuning, sequentially executing four clinical tasks: multi-label disease classification, lesion localization, anatomical segmentation, and radiology report generation. Experimental validation demonstrates competitive performance on ten CXR benchmarks under direct inference and fine-tuning settings. In a blinded evaluation of 160 historical reports from four medical centers, three radiologists rated 66\% of AI-generated reports as comparable to or surpassing original clinical reports in diagnostic clarity, highlighting the framework's translational potential. By establishing traceable diagnostic pathways from anatomical findings to conclusions, this work bridges the gap between AI technical metrics and clinical utility, advancing trustworthy AI systems in medical imaging.

Problem

• multi-task learning
• chest radiograph analysis
• interpretable AI
• clinical diagnosis
• medical imaging

Innovation

• interpretable AI
• multi-task learning
• chest X-ray analysis
• transformer architecture
• clinical report generation