Fundamental Limitation in Explaining AI

2026-05-23

AI Summary

This work addresses a fundamental challenge in artificial intelligence: the inherent tension among environmental complexity, system performance, interpretability, and complete faithfulness in AI explanations. We formally introduce and rigorously prove the “quadrilemma of AI explainability,” demonstrating that perfect faithfulness—i.e., an explanation fully capturing the true reasoning of a high-performing AI system—is theoretically unattainable in complex environments. Through formal modeling and mathematical derivation, we reveal the intrinsic trade-offs between an AI system’s capabilities and the fidelity of its explanations, showing that complete faithfulness must generally be sacrificed in practical settings. This study establishes theoretical limits for explainable AI and provides a foundational basis for trustworthy AI governance frameworks.

Abstract

While large-scale models such as LLMs and diffusion models have achieved practical success, public institutions have emphasized the importance of explainability in AI. Existing methods for explaining AI, however, are not designed to provide completely faithful explanations of the behavior of large-scale AI systems. Although a completely faithful and interpretable explanation of the behavior of an AI system might be useful for AI governance, it has not been known whether providing such an explanation is theoretically possible. In this paper, we mathematically prove a fundamental quadrilemma in explaining AI, stating that AI and its explanation cannot satisfy the following four conditions simultaneously: 1) the complexity of the operation environment, 2) the goodness of the AI's performance, 3) the interpretability of the AI's explanation, and 4) the complete faithfulness of the AI's explanation. This quadrilemma suggests that, in most applications where we cannot change the environment or sacrifice good AI performance and an interpretable explanation, we should give up complete faithfulness of explanations and should instead aim to explain only the parts that are important for applications. As a consequence, the quadrilemma implies that AI governance should be designed on the premise that the faithfulness of AI explanations is always incomplete.

Problem

• explainability
• faithfulness
• interpretability
• AI governance
• large-scale models

Innovation

• AI explainability
• quadrilemma
• faithfulness
• interpretability
• AI governance