Machine-learning competition to grade EEG background patterns in newborns with hypoxic-ischaemic encephalopathy

2025-08-27

arXiv.org

AI Summary

Automated EEG background pattern grading for neonatal hypoxic-ischemic encephalopathy (HIE) remains a clinically critical yet unsolved challenge due to limited generalizability of existing methods. Method: We established the first multicenter machine learning competition framework specifically designed for clinical EEG background grading, leveraging 353 hours of expert-annotated, anonymized, cross-center EEG data. A standardized pipeline—including training, validation, and an independent hold-out test set—was implemented to rigorously evaluate both handcrafted feature-based and deep learning models. Contribution/Results: All top-four models exhibited significant performance degradation on the independent test set, confirming poor generalizability across centers. Deep learning models demonstrated relatively greater robustness, underscoring the importance of large-scale, diverse datasets for clinical deployment. This framework advances reproducible, verifiable AI-driven assessment of neonatal brain function, setting a new benchmark for translational neurocritical care informatics.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Machine learning (ML) has the potential to support and improve expert performance in monitoring the brain function of at-risk newborns. Developing accurate and reliable ML models depends on access to high-quality, annotated data, a resource in short supply. ML competitions address this need by providing researchers access to expertly annotated datasets, fostering shared learning through direct model comparisons, and leveraging the benefits of crowdsourcing diverse expertise. We compiled a retrospective dataset containing 353 hours of EEG from 102 individual newborns from a multi-centre study. The data was fully anonymised and divided into training, testing, and held-out validation datasets. EEGs were graded for the severity of abnormal background patterns. Next, we created a web-based competition platform and hosted a machine learning competition to develop ML models for classifying the severity of EEG background patterns in newborns. After the competition closed, the top 4 performing models were evaluated offline on a separate held-out validation dataset. Although a feature-based model ranked first on the testing dataset, deep learning models generalised better on the validation sets. All methods had a significant decline in validation performance compared to the testing performance. This highlights the challenges for model generalisation on unseen data, emphasising the need for held-out validation datasets in ML studies with neonatal EEG. The study underscores the importance of training ML models on large and diverse datasets to ensure robust generalisation. The competition's outcome demonstrates the potential for open-access data and collaborative ML development to foster a collaborative research environment and expedite the development of clinical decision-support tools for neonatal neuromonitoring.

Problem

• Developing ML models to classify EEG background severity in newborns
• Addressing data scarcity through competition-based model development
• Evaluating model generalization on unseen neonatal EEG validation data

Innovation

• Used machine learning competition for EEG grading
• Evaluated top models on held-out validation dataset
• Emphasized large diverse datasets for model generalization