🤖 AI Summary
This work addresses the challenge of automatically extracting fine-grained communication behaviors from secure patient–clinician messages in alignment with the EPPC ontology structure, which requires simultaneous adherence to code/sub-code hierarchy constraints and precise textual grounding. To this end, the authors propose EPPC-OASIS, a novel method that explicitly aligns language model representations with the ontology through a Wasserstein alignment objective. The approach further incorporates an inference refinement mechanism comprising validation, self-consistency checks, and hybrid correction to enhance prediction coherence. Experiments on de-identified clinical dialogue corpora demonstrate that the method enables scalable structured extraction across multiple open-source large language models, achieving state-of-the-art performance with 77.13% Code+Sub-code F1 and 63.83% Triplet F1—improving upon strong baselines by 1.39 and 2.12 F1 points, respectively.
📝 Abstract
Secure patient-provider messages contain clinically important communication behaviors that are difficult to characterize manually at scale. The Electronic Patient-Provider Communication (EPPC) framework provides an ontology for coding these behaviors, but automated extraction remains challenging because predictions must preserve fine-grained code/sub-code structure while grounding annotations in message text. We developed EPPC-OASIS, an ontology-aware adaptation approach for structured EPPC extraction, and combined it with deployable inference-refinement procedures designed to improve the coherence of final annotations. EPPC-OASIS augments supervised fine-tuning with a Wasserstein alignment objective that encourages alignment between model representation neighborhoods and EPPC ontology-derived neighborhoods, while inference refinement uses verification, self-consistency, hybrid correction, and selection or ensembling to address residual prediction errors. We evaluated the framework on a de-identified corpus of secure patient-provider messages against prompting, supervised fine-tuning, preference-based, and robustness-oriented baselines across multiple open-weight language models. Across model families, the best deployable pipeline achieved 77.13% Code+Sub-code F1 and 63.83% Triplet F1, corresponding to modest but consistent absolute gains of +1.39 and +2.12 F1 points over the strongest supervised fine-tuning baseline. These results suggest that ontology-aware adaptation with structured inference refinement can support scalable retrospective EPPC mining, although external validation is needed before operational use.