An Information-Flow Perspective on Explainability Requirements: Specification and Verification

2025-09-01

AI Summary

Existing explainable systems face an inherent tension between explanation sufficiency and privacy preservation—how to formally guarantee that causal explanations (forward information flow) are provided to interactive agents while strictly bounding sensitive information leakage (backward information flow). Method: We propose the first unified verification framework modeling explainability and privacy as dual, opposing information flows. We introduce an extended epistemic temporal logic (ETL) integrated with counterfactual causal quantification to formalize explainability specifications. A model-checking algorithm is designed to automatically verify the joint satisfaction of explanation sufficiency and privacy constraints over finite-state systems. Contribution: We present the first logic-based specification language and verification tool that jointly enforces causal explainability and privacy boundaries. Evaluated on multiple benchmarks, our approach effectively distinguishes explainable from non-explainable systems, establishing a formally verifiable foundation for trustworthy multi-agent systems.

Technology Category

• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Reasoning under Uncertainty: Causality

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Explainable systems expose information about why certain observed effects are happening to the agents interacting with them. We argue that this constitutes a positive flow of information that needs to be specified, verified, and balanced against negative information flow that may, e.g., violate privacy guarantees. Since both explainability and privacy require reasoning about knowledge, we tackle these tasks with epistemic temporal logic extended with quantification over counterfactual causes. This allows us to specify that a multi-agent system exposes enough information such that agents acquire knowledge on why some effect occurred. We show how this principle can be used to specify explainability as a system-level requirement and provide an algorithm for checking finite-state models against such specifications. We present a prototype implementation of the algorithm and evaluate it on several benchmarks, illustrating how our approach distinguishes between explainable and unexplainable systems, and how it allows to pose additional privacy requirements.

Problem

• Specifying explainability requirements in multi-agent systems
• Verifying information flow against privacy guarantees
• Developing algorithms for checking explainability in finite-state models

Innovation

• Epistemic temporal logic for explainability specification
• Algorithm for finite-state model verification
• Quantifying counterfactual causes in multi-agent systems