Problem
Research questions and friction points this paper is trying to address.
legal investigation
AI agents
hypothesis evaluation
summary judgment
evidence reasoning
InvestigationWorlds: An Agentic Environment for Legal Investigation
🤖 AI Summary
This study addresses the challenge faced by AI agents in identifying court-adopted factual hypotheses from ambiguous evidence during legal investigations. The proposed method leverages frequently overlooked motions for summary judgment, utilizing real-world U.S. civil litigation data from PACER. Through an attorney-validated pipeline, the authors construct a multi-interpretation benchmark environment annotated with role labels, retaining only unique ground truths to simulate authentic legal investigation scenarios. The primary contribution is a novel evaluation framework for AI agents. An assessment of one hundred cases reveals that while agents can successfully retrieve relevant evidence, they frequently converge on incorrect hypotheses not adopted by the court. These findings expose critical limitations in complex legal reasoning capabilities, highlighting a significant gap between evidentiary retrieval and accurate judicial fact-finding in current AI systems.
📝 Abstract
We introduce InvestigationWorlds, an agentic environment for legal investigation. We build on an underused artifact of U.S. civil litigation: the summary judgment motion. This motion relies upon a record composed of real evidence exhibits, and results in a court-adopted hypothesis that is treated as ground truth for the purposes of deciding the motion. Each environment is built from a real U.S. Federal Court case retrieved from Public Access to Court Electronic Records (PACER) and augmented by an attorney-validated generation pipeline that synthesizes role-tagged documents around the original record. The resulting corpus admits multiple coherent factual readings, only one of which matches the court-adopted hypothesis. Evaluating on 100 cases, we find agents often commit to incorrect hypotheses despite retrieving relevant evidence, struggling to distinguish the court-adopted hypothesis from alternative hypotheses.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Agentic Environment
Legal Investigation
Summary Judgment Motion
Attorney-Validated Generation Pipeline
Ground Truth
🔎 Similar Papers
No similar papers found.
