Can large language models unlock discrete data in ophthalmic diagnostic reports?

2026-09-30

AI Summary

This study addresses the inefficiency and limited accuracy of structured information extraction from ophthalmic diagnostic reports. Leveraging GPT-4o, we compare JSON Schema-constrained generation with pure prompting strategies for automated information extraction across four categories of ophthalmology reports. A Python-based post-processing pipeline is integrated, and performance is evaluated against manual annotation baselines in terms of accuracy and processing time. Results demonstrate that the proposed approach achieves near-perfect accuracy while reducing extraction time by approximately 92% compared to manual annotation. Furthermore, the analysis reveals complementary strengths between the two strategies regarding numerical precision and formatting consistency. Overall, this work validates the efficiency and application potential of general-purpose large language models in processing domain-specific medical documents.

Abstract

Objective: To assess the accuracy and efficiency of a large language model (LLM) using two prompt strategies to extract structured data from ophthalmic diagnostic PDF reports. Methods: Twenty deidentified reports across four types (Visual Field, OCT Glaucoma Overview, OCT retinal nerve fiber layer Single Exam, and OCT Thickness Map; n = 5 each) were processed using two GPT-4o-assisted pipelines and compared with a reconciled manual ground truth. Schema-Constrained used Structured Output mode with a predefined JSON Schema; Prompt-Only used a detailed instruction prompt followed by Python conversion to JSON. Outcomes were value accuracy, formatting accuracy, and extraction time. Results: Schema-Constrained value accuracy was 100.00% for Visual Field and RNFL Single Exam, 97.45% for Glaucoma Overview, and 98.00% for Thickness Map; Prompt-Only achieved 100.00% across all four report types. Formatting accuracy was 100.00% for Schema-Constrained across all report types and 100.00% for Prompt-Only except RNFL Single Exam (90.14%). Mean extraction time was 56.51 s per report for manual review versus 5.04 s for Schema-Constrained and 4.70 s for Prompt-Only, an approximately 92% reduction. Conclusions: In this small proof-of-concept dataset, general-purpose LLM-assisted pipelines extracted structured data from ophthalmic diagnostic PDFs with high accuracy and substantially reduced processing time. Prompt-Only achieved the highest value accuracy, while Schema-Constrained produced schema-compliant output with 100% formatting accuracy. These complementary strengths support further evaluation of hybrid, validation-aware workflows for research and clinical data abstraction.

Problem

• Large Language Models
• Ophthalmic Diagnostic Reports
• Data Extraction
• Structured Data
• PDF Reports

Innovation

• Large Language Models
• Data Extraction
• Prompt Engineering
• Structured Output
• Ophthalmic Reports