🤖 AI Summary
This study addresses recurring failure modes and post-certification security degradations in Common Criteria (CC) evaluations. By systematically integrating the ISO/IEC 15408 standard with the Common Evaluation Methodology (CEM), this work pioneers an evaluator-centric, cross-vendor taxonomy of lifecycle failures alongside a comprehensive root cause analysis. Furthermore, it proposes an evaluability-by-design framework and associated guidelines tailored for CC assessments, while systematically identifying unresolved challenges in critical domains such as cloud computing and artificial intelligence. The primary contribution of this research lies in bridging the existing gap in the classification of CC evaluation failure modes. Ultimately, it provides both theoretical foundations and practical guidance for enhancing the security evaluability and certification effectiveness of complex systems.
📝 Abstract
The Common Criteria (CC; ISO/IEC 15408) is the principal international framework for evaluating the security of IT products, and its certificates gate procurement across government, defense, and regulated industry. Yet evaluations routinely stall, fail, or yield certificates whose assurances do not survive real-world deployment. Prior work has approached this from two directions: economic critiques of misaligned incentives and "security theatre"; and recent data-driven studies that quantify vulnerabilities in already-certified products. Comparatively little systematizes the problem from the evaluator's operational vantage - recurring, cross-vendor failures against the Common Criteria work units themselves. This paper presents a systematization of knowledge (SoK) of failure modes in Common Criteria product evaluation. Drawing on the standard, the Common Evaluation Methodology (ISO/IEC 18045), public NIAP Protection Profiles, and published Security Targets and Certification Reports, we organize recurring failures into a lifecycle taxonomy spanning Security Target scoping, Protection Profile conformance, assurance evidence, cryptographic requirements, functional testing, vulnerability analysis, operational guidance, and post-certification configuration drift. We analyze their root causes, derive a design-for-evaluability framework that product teams can apply before evaluation begins, and identify open problems for cloud, continuous-delivery, and AI-enabled systems.