ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations

📅 2026-09-01
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
本文提出ClaimReceipt方法,通过绑定类型化交易证据和签名实验清单来验证代理评估中的证据充分性和覆盖率问题。
📝 Abstract
Agent evaluations face two distinct evidentiary questions: whether a reported claim is recomputable from retained evidence (sufficiency), and whether the retained records cover the committed experiment set (coverage). Generic logs and hash-linked transcripts answer neither reliably. We introduce ClaimReceipt, a claim-relative receipt specification and selective verifier that binds typed transaction evidence to a signed experiment manifest and returns PASS, INVALID, or INCONCLUSIVE per claim. We freeze the specification before implementation (SHA-256 18d109...b81). On 1,392 historical buyer--seller records, a CR-2 verifier reproduces all five manually labeled audit verdicts, exactly replays 600 deterministic and 792 post-generation records, makes every one of 13 declared field groups non-redundant under tested ablations, and returns the expected result on 11/11 semantic faults with 0/8 false positives. We then run a separate prospective CR-3 epoch: 30 assignments are committed before inference, terminal receipts are signed and chained, and private evidence is encrypted for an auditor. Complete evidence yields coverage and accounting PASS; withholding one terminal receipt returns INCONCLUSIVE_COVERAGE, while withholding all private openings preserves coverage and protocol verification but makes economic claims inconclusive, exactly matching a preregistered prediction. Receipt instrumentation adds 0.021% of model-inference time and 9.9 KB per transaction. A specification-legibility probe indicates that our own frozen specification is not yet unambiguous to an independent reader. Claim verification therefore requires both claim-sufficient evidence and a committed universe against which omissions become visible.
Problem

Research questions and friction points this paper is trying to address.

evidence sufficiency
coverage
agent evaluations
Innovation

Methods, ideas, or system contributions that make the work stand out.

ClaimReceipt
sufficiency
coverage
selective verifier
typed transaction evidence
P
Peiying Zhu
Blossom AI
S
Sidi Chang
Blossom AI Labs