Automated Radiology Report Generation Based on Topic-Keyword Semantic Guidance

2025-09-13

AI Summary

Existing radiology report generation methods neglect semantic knowledge embedded in historical reports, resulting in insufficient prior information and limited clinical accuracy of generated reports. To address this, we propose the Topic-Keyword Semantic Guidance (TKSG) framework—the first to jointly integrate topic modeling and keyword extraction for hierarchical semantic guidance: global topic vectors constrain the macro-structure of reports, while local semantic attention mechanisms focus on critical diagnostic entities. Furthermore, TKSG leverages BiomedCLIP for cross-modal retrieval of relevant historical cases to inform multimodal decoding. Evaluated on IU X-Ray and MIMIC-CXR, TKSG achieves statistically significant improvements (p < 0.01) in BLEU, CIDEr, and clinically oriented metrics. Generated reports exhibit greater alignment with radiologists’ linguistic conventions and demonstrate tangible clinical applicability.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Automated radiology report generation is essential in clinical practice. However, diagnosing radiological images typically requires physicians 5-10 minutes, resulting in a waste of valuable healthcare resources. Existing studies have not fully leveraged knowledge from historical radiology reports, lacking sufficient and accurate prior information. To address this, we propose a Topic-Keyword Semantic Guidance (TKSG) framework. This framework uses BiomedCLIP to accurately retrieve historical similar cases. Supported by multimodal, TKSG accurately detects topic words (disease classifications) and keywords (common symptoms) in diagnoses. The probabilities of topic terms are aggregated into a topic vector, serving as global information to guide the entire decoding process. Additionally, a semantic-guided attention module is designed to refine local decoding with keyword content, ensuring report accuracy and relevance. Experimental results show that our model achieves excellent performance on both IU X-Ray and MIMIC-CXR datasets. The code is available at https://github.com/SCNU203/TKSG.

Problem

• Automated radiology report generation for clinical efficiency
• Leveraging historical report knowledge for accurate diagnostics
• Detecting disease classifications and symptoms through multimodal guidance

Innovation

• Topic-Keyword Semantic Guidance framework
• BiomedCLIP for historical case retrieval
• Semantic-guided attention module decoding