ChexFract: From General to Specialized -- Enhancing Fracture Description Generation

2025-11-11

AI Summary

In medical AI, general-purpose radiology report generation models exhibit severe limitations in detecting and describing rare yet critical pathologies—such as fractures. To address this, we propose the first dedicated vision-language model for fracture diagnosis in chest X-rays: leveraging transfer learning from MAIRA-2 and CheXagent visual encoders, coupled with a fracture-specific textual decoder, enabling end-to-end training. We systematically analyze how fracture type, anatomical location, and patient age affect model performance. Experiments demonstrate statistically significant improvements (p < 0.01) in fracture description accuracy over general-purpose baselines, particularly for subtle and multifocal fractures. To foster reproducibility and advancement, we fully open-source the code, model weights, and a standardized evaluation benchmark—establishing both a reusable tool and a new benchmark for precise automated reporting of rare pathologies.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Multi-modal Vision
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Generating accurate and clinically meaningful radiology reports from chest X-ray images remains a significant challenge in medical AI. While recent vision-language models achieve strong results in general radiology report generation, they often fail to adequately describe rare but clinically important pathologies like fractures. This work addresses this gap by developing specialized models for fracture pathology detection and description. We train fracture-specific vision-language models with encoders from MAIRA-2 and CheXagent, demonstrating significant improvements over general-purpose models in generating accurate fracture descriptions. Analysis of model outputs by fracture type, location, and age reveals distinct strengths and limitations of current vision-language model architectures. We publicly release our best-performing fracture-reporting model, facilitating future research in accurate reporting of rare pathologies.

Problem

• Enhancing fracture description generation in chest X-rays
• Addressing inadequate rare pathology detection in general models
• Developing specialized vision-language models for fracture reporting

Innovation

• Specialized vision-language models for fracture detection
• Leveraged MAIRA-2 and CheXagent encoder architectures
• Enhanced fracture description accuracy over general models