Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices

2025-06-18

AI Summary

AI/ML-enabled medical devices face pre-market cybersecurity risks stemming from model complexity, pervasive interconnectivity, and underlying software vulnerabilities. Method: We propose the first proactive assessment framework integrating systems safety theory with data-driven analysis—leveraging public recall and adverse event databases, multi-dimensional threat modeling, extended System-Theoretic Process Analysis (STPA), and model interpretability validation techniques. Contribution/Results: Our framework embeds cybersecurity rigorously into the device design lifecycle and delivers an actionable toolkit enabling security analysts to quantitatively characterize ML model failure modes under adversarial conditions. Empirical evaluation demonstrates substantial improvement in design-phase security resilience. The framework provides regulators—including the FDA—with a practical, implementable risk assessment paradigm, advancing AI medical device security from reactive, post-deployment incident response toward proactive, intrinsic safety-by-design.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Safety and Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Security and privacy of machine learning and AI applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The integration of AI/ML into medical devices is rapidly transforming healthcare by enhancing diagnostic and treatment facilities. However, this advancement also introduces serious cybersecurity risks due to the use of complex and often opaque models, extensive interconnectivity, interoperability with third-party peripheral devices, Internet connectivity, and vulnerabilities in the underlying technologies. These factors contribute to a broad attack surface and make threat prevention, detection, and mitigation challenging. Given the highly safety-critical nature of these devices, a cyberattack on these devices can cause the ML models to mispredict, thereby posing significant safety risks to patients. Therefore, ensuring the security of these devices from the time of design is essential. This paper underscores the urgency of addressing the cybersecurity challenges in ML-enabled medical devices at the pre-market phase. We begin by analyzing publicly available data on device recalls and adverse events, and known vulnerabilities, to understand the threat landscape of AI/ML-enabled medical devices and their repercussions on patient safety. Building on this analysis, we introduce a suite of tools and techniques designed by us to assist security analysts in conducting comprehensive premarket risk assessments. Our work aims to empower manufacturers to embed cybersecurity as a core design principle in AI/ML-enabled medical devices, thereby making them safe for patients.

Problem

• Address cybersecurity risks in AI/ML medical devices
• Analyze threats from recalls, vulnerabilities, and adverse events
• Develop tools for premarket security risk assessment

Innovation

• Systems-Theoretic analysis for ML medical devices
• Data-driven security assessment premarket phase
• Tool suite for comprehensive risk analysis