From Evidence-Based Medicine to Knowledge Graph: Retrieval-Augmented Generation for Sports Rehabilitation and a Domain Benchmark

2026-01-01

arXiv.org

AI Summary

This study addresses a critical gap in current retrieval-augmented generation (RAG) approaches, which often neglect evidence-based medicine principles and lack alignment with the PICO (Population, Intervention, Comparison, Outcome) framework or assessment of evidence hierarchy. To bridge this gap, the authors propose the first RAG system that integrates PICO structure into both knowledge graph construction and retrieval, complemented by a Bayesian-inspired, weight-free heuristic reranking algorithm that is sensitive to levels of evidence. The work introduces a domain-specific knowledge graph for sports rehabilitation comprising 357,000 nodes and a benchmark of 1,637 question-answer pairs. Experimental results demonstrate superior performance on key metrics, with expert evaluations yielding scores of 4.66–4.84 on a 5-point scale, significantly enhancing answer accuracy and clinical relevance.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In medicine, large language models (LLMs) increasingly rely on retrieval-augmented generation (RAG) to ground outputs in up-to-date external evidence. However, current RAG approaches focus primarily on performance improvements while overlooking evidence-based medicine (EBM) principles. This study addresses two key gaps: (1) the lack of PICO alignment between queries and retrieved evidence, and (2) the absence of evidence hierarchy considerations during reranking. We present a generalizable strategy for adapting EBM to graph-based RAG, integrating the PICO framework into knowledge graph construction and retrieval, and proposing a Bayesian-inspired reranking algorithm to calibrate ranking scores by evidence grade without introducing predefined weights. We validated this framework in sports rehabilitation, a literature-rich domain currently lacking RAG systems and benchmarks. We released a knowledge graph (357,844 nodes and 371,226 edges) and a reusable benchmark of 1,637 QA pairs. The system achieved 0.830 nugget coverage, 0.819 answer faithfulness, 0.882 semantic similarity, and 0.788 PICOT match accuracy. In a 5-point Likert evaluation, five expert clinicians rated the system 4.66-4.84 across factual accuracy, faithfulness, relevance, safety, and PICO alignment. These findings demonstrate that the proposed EBM adaptation strategy improves retrieval and answer quality and is transferable to other clinical domains. The released resources also help address the scarcity of RAG datasets in sports rehabilitation.

Problem

• Evidence-Based Medicine
• Retrieval-Augmented Generation
• PICO Framework
• Evidence Hierarchy
• Knowledge Graph

Innovation

• Retrieval-Augmented Generation
• Evidence-Based Medicine
• Knowledge Graph
• PICO Framework
• Bayesian Reranking