RecallRisk-BERT: A Multi-Task Framework for Post-Report Medical Device Recall Triage

2026-06-25

AI Summary

This study addresses the challenge of simultaneously classifying recall severity levels (Class I–III) and identifying nine consolidated root causes in the face of rapidly increasing FDA medical device recall reports—a task inadequately handled by existing approaches that treat prediction and attribution in isolation. To this end, we propose RecallRisk-BERT, the first multitask learning framework that integrates unstructured recall narrative text, encoded via PubMedBERT, with structured features such as product codes and regulation numbers. Our model incorporates categorical feature embeddings within a LightGBM-enhanced architecture, achieving 0.963 accuracy and 0.856 macro F1-score in severity prediction. The multitask formulation significantly outperforms single-task baselines, yielding risk rankings highly aligned with actual recall severities (Spearman ρ = 0.983, p < 0.001), thereby enabling automated recall triage and supporting regulatory decision-making.

Abstract

Medical device recalls are a critical regulatory mechanism for protecting patient safety. The growing volume of FDA recall records presents challenges in post-report recall triage, severity assessment, and root-cause interpretation. Existing studies mostly address recall occurrence prediction or root-cause analysis separately, while joint modeling of recall severity and root-cause categories has received limited attention. We develop an automated recall triage framework using 54,165 FDA medical device recall records from openFDA, covering the period from 2002 to October 2025. We first evaluate classical machine learning and boosting-based models for recall severity and root-cause category prediction. We then develop RecallRisk-BERT, a multi-task model that combines PubMedBERT-based textual representations of recall narratives with embedding-based representations of structured categorical features, including product code, regulation number, and medical specialty. The model simultaneously predicts recall severity (Class I/II/III) and a consolidated root-cause category (9 classes). Performance was evaluated using accuracy, macro-averaged precision, recall, F1-score, and ROC-AUC. In single-task severity prediction, our LightGBM-based text--tabular configuration achieved the strongest performance, with an accuracy of 0.963, macro-F1 of 0.856, and ROC-AUC of 0.974. In the multi-task setting, RecallRisk-BERT substantially outperformed the single-task PubMedBERT baseline. Model-derived risk rankings were strongly consistent with observed root-cause severity patterns (rho = 0.983, p = 1.936e-6). These findings indicate that text--tabular learning can support scalable post-report recall triage, regulatory decision support, and model-based root-cause risk analysis.

Problem

• medical device recall
• recall triage
• severity assessment
• root-cause analysis
• multi-task learning

Innovation

• multi-task learning
• RecallRisk-BERT
• text-tabular fusion
• medical device recall
• root-cause analysis