Explainability by design: an experimental analysis of the legal coding process

2025-05-03

AI Summary

This paper addresses the interpretable encoding of legal texts into deontic defeasible logic rules. We propose a novel method wherein interpretability arises intrinsically from the encoding process itself: normative text fragments are systematically translated into formal logical rules, and multi-dimensional test scenarios are designed to empirically validate semantic correctness. A key innovation is the introduction of a “depth” metric—quantifying the hierarchical complexity of legal citations—integrated with empirical experiments and regression modeling to build a predictive model of encoding time. Our study provides the first empirical evidence that text length, domain expertise, coder experience, and citation depth significantly impact encoding efficiency. The approach ensures logical fidelity while unifying transparency and efficiency in legal knowledge engineering. It establishes a reproducible, evaluable technical pathway for explainable AI–driven legal automation.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Behind a set of rules in Deontic Defeasible Logic, there is a mapping process of normative background fragments. This process goes from text to rules and implicitly encompasses an explanation of the coded fragments. In this paper we deliver a methodology for extit{legal coding} that starts with a fragment and goes onto a set of Deontic Defeasible Logic rules, involving a set of extit{scenarios} to test the correctness of the coded fragments. The methodology is illustrated by the coding process of an example text. We then show the results of a series of experiments conducted with humans encoding a variety of normative backgrounds and corresponding cases in which we have measured the efforts made in the coding process, as related to some measurable features. To process these examples, a recently developed technology, Houdini, that allows reasoning in Deontic Defeasible Logic, has been employed. Finally we provide a technique to forecast time required in coding, that depends on factors such as knowledge of the legal domain, knowledge of the coding processes, length of the text, and a measure of extit{depth} that refers to the length of the paths of legal references.

Problem

• Develops a legal coding methodology for Deontic Defeasible Logic rules
• Measures human effort in encoding normative backgrounds and cases
• Provides a technique to forecast coding time based on factors

Innovation

• Methodology for legal coding in Deontic Defeasible Logic
• Houdini technology for reasoning in Deontic Defeasible Logic
• Technique to forecast coding time based on factors