🤖 AI Summary
This study addresses reliability challenges in composite AI systems arising from cascading errors, silent degradation, and coordination failures at component boundaries. Drawing on 150 production incidents, it presents the first systematic fault taxonomy and proposes a catalog of resilience patterns encompassing five failure categories, including retrieval and generation faults. The effectiveness of mitigation strategies—specifically circuit breakers, output quality gates, and component isolation—is validated through fault injection experiments. Results demonstrate that circuit breakers reduce cascading propagation by 89%, quality gates detect 73% of silent degradations, and combining multiple patterns decreases mean time to recovery (MTTR) by 71%. These findings provide practitioners with an empirically grounded resource for reliability engineering in complex AI architectures.
📝 Abstract
Deploying compound AI systems reliably and safely requires understanding failure modes that emerge at component boundaries, not within individual models. Cascading errors propagate across component boundaries, silent quality degradation evades standard monitoring, and coordination failures yield incorrect collective behavior from individually correct parts. We analyze 150 production incident reports from open-source compound AI projects and anonymized enterprise deployments to construct a taxonomy of 23 failure modes organized into five categories: retrieval failures, generation failures, tool failures, orchestration failures, and integration failures. For each category, we propose resilience patterns with measured effectiveness from controlled fault injection experiments. Circuit breakers reduce cascade propagation by 89%, output quality gates catch 73% of silent degradation before user impact, and component isolation reduces blast radius by 64%. Systems implementing three or more resilience patterns from our catalog reduce mean-time-to-recovery (MTTR) by 71% compared to unstructured monitoring baselines. We release the incident taxonomy and pattern catalog as a practitioner resource.