On AI Safety and Security Technical Debt in Engineering AI-Enabled Systems

2026-07-25

AI Summary

This study addresses the critical yet often overlooked issue of AI safety-related technical debt, which arises from vulnerabilities in data, models, and system architectures within high-stakes AI applications and can severely compromise system trustworthiness. For the first time, the paper explicitly frames AI safety through the lens of technical debt, leveraging the AI TRiSM principles to conduct a systematic literature review that identifies 31 distinct types of AI safety technical debt. It proposes a unified AITD-MAP framework organized around seven root-cause categories, links these debts to 18 key trustworthy AI concerns, and offers 34 actionable mitigation guidelines. This framework enables visualization, root-cause tracing, and lifecycle-wide governance of AI safety debt, substantially enhancing its manageability in AI engineering practice.

Abstract

Artificial intelligence (AI) systems are increasingly deployed in high-stakes domains such as healthcare, autonomous driving, finance, and education. While these systems offer powerful data-driven and adaptive capabilities, their complexity, rapid evolution, and dependence on dynamic data pipelines introduce new forms of engineering liability collectively referred to as AI Technical Debts (AITDs). AITDs arise from root causes spanning data governance, model implementation, algorithm design, architectural decisions, operational processes, documentation practices, and testing adequacy. Unlike conventional technical debt, many AITDs are latent and propagate across tightly coupled AI pipelines, leading to maintenance challenges, reliability degradation, and heightened safety or security risks. Guided by the principles of AI Trust, Risk, and Security Management (AI TRiSM), this study reinterprets technical debt through the interconnected dimensions of trustworthiness, focusing on AI safety and security technical debts. We conduct a systematic review of 60 primary studies and identify 31 distinct types of AITD, which are organized into a root-cause-oriented taxonomy comprising seven classes. The analysis examines how these debts map to 18 trust-related concerns, including 6 safety hazards and 12 security vulnerabilities. To support mitigation, the review synthesizes 34 actionable guidelines (8 safety and 26 security) targeting the prevention, detection, and reduction of AITDs across the AI lifecycle. Building on these findings, we introduce AITD-MAP, an integrated framework that connects the AITD taxonomy, quality and risk impacts, and mitigation strategies into a unified structure for risk-aware AI engineering. The framework aims to assist AI software engineers in making AI safety and security technical debts visible, understanding their root causes, and mitigating their presence.

Problem

• AI Technical Debt
• AI Safety
• AI Security
• Trustworthiness
• Risk Management

Innovation

• AI Technical Debt
• AI Safety
• AI Security
• AITD-MAP
• AI TRiSM