LLM-Orchestrated Conformance Checking in Stroke Care Without Computer-Interpretable Guidelines

2026-06-08

AI Summary

This work proposes a modular framework for assessing compliance with clinical care pathways in the absence of predefined computer-interpretable guidelines (CIGs), which are often unavailable in real-world settings. By orchestrating multiple large language models (LLMs), the approach automatically extracts patient trajectories and normative rules directly from unstructured clinical records and textual guidelines, then translates them into executable scripts to quantify adherence. Integrating LLM orchestration, natural language processing, and event log analysis, the method was evaluated on hundreds of stroke patients at Alexandria Hospital’s neurology department. Results demonstrated that over 86% of care processes aligned with guideline recommendations, confirming the framework’s feasibility and clinical applicability.

Abstract

Objective: Conformance checking in healthcare seeks to assess whether patient care pathways adhere to clinical guidelines. However, its practical application often depends on the availability of formal, machine-interpretable representations of guidelines, such as Computer-Interpretable Guidelines (CIGs), which are seldom available in real-world clinical settings. Methods: This work introduces a modular framework based on the orchestration of Large Language Models (LLMs) to support medical conformance checking directly from unstructured clinical and guideline texts, without requiring predefined CIGs. The proposed architecture integrates multiple LLMs and supporting components to extract patient traces from clinical discharge letters, identify normative rules from textual clinical guidelines, translate these rules into executable scripts, and compute a Trace Conformance Indicator to quantify compliance within the event log. Results: The framework was implemented and evaluated in the stroke care domain at the neurological ward of Alessandria Hospital. Hundreds of patient traces were automatically extracted from hospital data and assessed against 50 rules derived from the reference guideline. The analysis showed that more than 86\% of the available traces were conformant. Conclusion: The results demonstrate the feasibility of using orchestrated LLMs for practical healthcare conformance analysis. At the same time, the study provides evidence of a high level of adherence to stroke care guidelines at Alessandria Hospital.

Problem

• conformance checking
• clinical guidelines
• Computer-Interpretable Guidelines
• healthcare compliance
• stroke care

Innovation

• LLM orchestration
• conformance checking
• clinical guidelines
• stroke care
• trace extraction