Authoring and Management of Transparent Research Integrity Assessments of Randomised Clinical Trial Publications Using LLM-assisted Tools and Provenance Knowledge Graphs

2026-08-07

AI Summary

Current assessments of the integrity of randomized controlled trials (RCTs) rely heavily on manual processes that are complex, subjective, and prone to inconsistency, thereby compromising the quality of evidence-based guidelines. To address this limitation, this work proposes INSPECT-AI, a novel framework that integrates large language models (LLMs) with a knowledge graph grounded in the RIPE-O ontology (RIPE-KG) to automate integrity evaluation, standardize semantic interpretation, and enable full auditability. The authors constructed a RIPE-KG comprising 95 RCTs annotated by experts across 140 assessment criteria and demonstrated that LLM-augmented evaluation significantly enhances efficiency, inter-rater consistency, and traceability. This approach establishes a transparent, reproducible paradigm for evidence synthesis in systematic reviews and guideline development.

Abstract

Systematic reviews of Randomised Controlled Trials (RCTs) are routinely used as evidence for clinical care guidelines. Such evidence has to meet high research integrity standards to prevent low quality or false research outputs influencing the clinical care. However, assessing research integrity of published RCTs is a complex process requiring manual effort, and potentially resulting in diverse opinions of the human assessors. This paper describes INSPECT-AI, an LLM-based interactive tool that assists human reviewers with research integrity assessments of published RCTs based on the community approved INSPECT-SR framework, and the Research Integrity Provenance and Evidence ontology (RIPE-O) for documenting the provenance of the assessment process. In addition, we present the Research Integrity Provenance and Evidence knowledge graph (RIPE-KG), an initial set of 140 expert research integrity assessments of 95 RCT publications generated by INSPECT-AI and described using RIPE-O.

Problem

• research integrity
• randomised controlled trials
• systematic reviews
• evidence-based clinical guidelines
• assessment consistency

Innovation

• LLM-assisted assessment
• research integrity
• provenance knowledge graph
• ontology
• randomised clinical trials