OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression

2026-10-08

AI Summary

This study addresses the challenges of multimodal fusion, limited interpretability, and time-consuming manual assessment in predicting knee osteoarthritis progression by proposing an autonomous multi-agent framework. The system integrates MRI, X-ray, and clinical expert agents, achieving automated evaluation through large language model tool calling and retrieval-augmented generation. Furthermore, it introduces a novel uncertainty-driven human-AI collaboration mechanism that enables physicians to review intermediate findings and trigger recomputation, significantly enhancing decision transparency via cross-modal conflict analysis. Evaluated on the FNIH cohort, the framework achieves AUROCs of 0.80 and 0.68 for structural and pain progression prediction, respectively. Case studies further validate the effectiveness of the interactive review process.

Abstract

Knee osteoarthritis (KOA) progression prediction can support patient monitoring, requiring the integration of multimodal data and multidomain expertise. Moreover, isolated risk estimates provide limited insight underlying a prediction. To automate the progression assessment workflow and reduce manual effort while providing interpretable findings and supporting evidence, we present OA-MAP, an autonomous multi-agent framework for evidence-grounded assessment of structural and pain progression in KOA. The system incorporates modality-specific agents including MRI, X-ray, and clinical agents, together with a coordinator agent. This framework can autonomously recruit specialist agents, select tools for prediction and analysis, and retrieve literature as external evidence based on user request and available patient information. An uncertainty-informed human-in-the-loop mechanism enables clinicians to review and correct intermediate findings, triggering recomputation of affected results. We evaluate the prediction models using 600 participants from the FNIH Osteoarthritis Biomarkers Consortium cohort. On the test set of 100 participants, the fusion models achieve AUROCs of 0.80 for structural progression and 0.68 for pain progression. A case study illustrates how OA-MAP combines risk estimates with intermediate findings, cross-modal conflicts, literature support, and uncertainty indicators to support interactive review.

Problem

• Knee Osteoarthritis
• Progression Prediction
• Multimodal Data
• Interpretability
• Evidence-Grounded

Innovation

• Multi-Agent Framework
• Multimodal Fusion
• Interpretable Prediction
• Human-in-the-Loop
• Knee Osteoarthritis