LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning

2025-05-29

AI Summary

To address the challenges of complex medical terminology and clinical context modeling in automated chest X-ray report generation, this paper proposes a lightweight, efficient multimodal framework. Our method introduces a novel dual-stream architecture integrating a DenseNet-121 image encoder with the LLaMA 3.1 language model, coupled with a QLoRA-based low-rank adaptation strategy specifically designed for medical text generation—enabling fine-grained image–text alignment and interpretable reasoning. The approach balances clinical accuracy and deployment efficiency: on the IU X-ray dataset, it achieves ROUGE-L = 0.433 and METEOR = 0.336, significantly outperforming existing state-of-the-art models. Moreover, its parameter-efficient design facilitates rapid deployment in low-resource clinical settings.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Computer Vision: Multi-modal VisionNatural Language Processing: Language Grounding & Multi-modal NLP

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Automated radiology report generation holds significant potential to reduce radiologists' workload and enhance diagnostic accuracy. However, generating precise and clinically meaningful reports from chest radiographs remains challenging due to the complexity of medical language and the need for contextual understanding. Existing models often struggle with maintaining both accuracy and contextual relevance. In this paper, we present LLaMA-XR, a novel framework that integrates LLaMA 3.1 with DenseNet-121-based image embeddings and Quantized Low-Rank Adaptation (QLoRA) fine-tuning. LLaMA-XR achieves improved coherence and clinical accuracy while maintaining computational efficiency. This efficiency is driven by an optimization strategy that enhances parameter utilization and reduces memory overhead, enabling faster report generation with lower computational resource demands. Extensive experiments conducted on the IU X-ray benchmark dataset demonstrate that LLaMA-XR outperforms a range of state-of-the-art methods. Our model achieves a ROUGE-L score of 0.433 and a METEOR score of 0.336, establishing new performance benchmarks in the domain. These results underscore LLaMA-XR's potential as an effective and efficient AI system for automated radiology reporting, offering enhanced clinical utility and reliability.

Problem

• Generating precise radiology reports from chest radiographs
• Maintaining accuracy and contextual relevance in reports
• Reducing computational resource demands for report generation

Innovation

• Integrates LLaMA 3.1 with DenseNet-121 embeddings
• Uses QLoRA fine-tuning for computational efficiency
• Optimizes parameter utilization and reduces memory overhead