DAR: Deontic Reasoning with Agentic Harnesses

2026-06-03

AI Summary

This work addresses the challenge that large language models often fail to reason accurately over lengthy and cross-referential deontic rules due to difficulties in precisely locating relevant provisions. To overcome this limitation, the paper proposes the Deontic Agent Reasoning (DAR) framework, which introduces an agent-based interaction mechanism enabling models to actively and selectively retrieve and dynamically apply legal provisions on demand, thereby transcending the constraints of static context dependence. Built upon large language models and equipped with diverse interactive interfaces, DAR facilitates flexible rule invocation. Experimental results demonstrate that DAR substantially improves reasoning performance on the challenging subset of DeonticBench, confirming its effectiveness, although gains are limited for weaker models on numerical tasks and come with additional computational overhead.

Abstract

Deontic reasoning is the task of answering questions by applying explicit rules and policies to case-specific facts, for example computing tax liability under a statute or determining the outcome of an immigration appeal. A key technical challenge for LLM-based deontic reasoning is that the relevant ruleset can be long and cross-referenced, so models may still fail to locate the rules needed for a particular reasoning step. We introduce Deontic Agentic Reasoning (DAR), an agentic reasoning setup in which the model interacts with the statutes on demand. We evaluate DAR under multiple harnesses on hard subsets of DeonticBench. Across these settings, we find that agentic harnesses can push the frontier on deontic reasoning tasks, but improvements are not uniform: weaker models often degrade on numerical tasks while consuming far more tokens.

Problem

• deontic reasoning
• large language models
• rule-based reasoning
• statutory interpretation
• cross-referenced rules

Innovation

• Deontic Reasoning
• Agentic Reasoning
• Rule-based QA
• Statutory Interpretation
• LLM Reasoning