When Secure Isn't: Assessing the Security of Machine Learning Model Sharing

2025-09-08

AI Summary

This study systematically assesses the security posture of mainstream machine learning model sharing frameworks and platforms, revealing three core issues: pervasive weaknesses in built-in security mechanisms, excessive delegation of security responsibility to end users, and widespread user security misconceptions. Employing a mixed-method approach—comprising security auditing, zero-day vulnerability discovery, and empirical user surveys—we identify six critical vulnerabilities enabling arbitrary code execution, thereby refuting the misconception that model file formats are inherently secure. To our knowledge, this is the first empirical demonstration that the current ecosystem lacks defense-in-depth, that platform security claims are frequently misleading, and that users over-rely on default configurations while underestimating risks associated with untrusted model loading. We propose a holistic set of security enhancements spanning technical design, governance policies, and user education—providing both theoretical foundations and actionable pathways toward trustworthy model-sharing infrastructure.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rise of model-sharing through frameworks and dedicated hubs makes Machine Learning significantly more accessible. Despite their benefits, these tools expose users to underexplored security risks, while security awareness remains limited among both practitioners and developers. To enable a more security-conscious culture in Machine Learning model sharing, in this paper we evaluate the security posture of frameworks and hubs, assess whether security-oriented mechanisms offer real protection, and survey how users perceive the security narratives surrounding model sharing. Our evaluation shows that most frameworks and hubs address security risks partially at best, often by shifting responsibility to the user. More concerningly, our analysis of frameworks advertising security-oriented settings and complete model sharing uncovered six 0-day vulnerabilities enabling arbitrary code execution. Through this analysis, we debunk the misconceptions that the model-sharing problem is largely solved and that its security can be guaranteed by the file format used for sharing. As expected, our survey shows that the surrounding security narrative leads users to consider security-oriented settings as trustworthy, despite the weaknesses shown in this work. From this, we derive takeaways and suggestions to strengthen the security of model-sharing ecosystems.

Problem

• Assessing security risks in ML model sharing frameworks
• Evaluating protection mechanisms for shared machine learning models
• Investigating user perceptions of security in model-sharing ecosystems

Innovation

• Evaluated security of model-sharing frameworks and hubs
• Uncovered six 0-day vulnerabilities in security settings
• Debunked misconceptions about solved model-sharing security