🤖 AI Summary
This work proposes a graph neural network–based approach to support compliance and safety verification by analyzing the structural soundness and source authenticity of assurance cases. For the first time, assurance cases are systematically modeled as textual attributed graphs, enabling a joint learning framework that performs link prediction—assessing structural coherence—and human–machine generated case classification—discriminating origin authenticity. The method reveals significant differences in hierarchical linkage patterns between large language model–generated cases and those authored by humans. Experimental results on real-world datasets demonstrate the effectiveness and novelty of the proposed framework, achieving a ROC-AUC of 0.760 for link prediction and an F1 score of 0.94 for human–machine case classification.
📝 Abstract
An assurance case is a structured argument document that justifies claims about a system's requirements or properties, which are supported by evidence. In regulated domains, these are crucial for meeting compliance and safety requirements to industry standards. We propose a graph diagnostic framework for analysing the structure and provenance of assurance cases. We focus on two main tasks: (1) link prediction, to learn and identify connections between argument elements, and (2) graph classification, to differentiate between assurance cases created by a state-of-the-art large language model and those created by humans, aiming to detect bias. We compiled a publicly available dataset of assurance cases, represented as graphs with nodes and edges, supporting both link prediction and provenance analysis. Experiments show that graph neural networks (GNNs) achieve strong link prediction performance (ROC-AUC 0.760) on real assurance cases and generalise well across domains and semi-supervised settings. For provenance detection, GNNs effectively distinguish human-authored from LLM-generated cases (F1 0.94). We observed that LLM-generated assurance cases have different hierarchical linking patterns compared to human-authored cases. Furthermore, existing GNN explanation methods show only moderate faithfulness, revealing a gap between predicted reasoning and the true argument structure.