Logical Consistency Between Disagreeing Experts and Its Role in AI Safety

2025-10-01

AI Summary

Evaluating decision consistency among experts (or classifiers) in unsupervised settings—without ground-truth labels or prior knowledge—remains challenging. Method: We propose a logic-based evaluation framework that models expert scores as an integer linear programming problem, integrating statistical alignment signals with formal logical constraints—including inequality constraints and a novel, universally applicable linear equality axiom for arbitrary finite test sets. This axiom ensures logical satisfiability of collective judgments. Contribution/Results: Our framework introduces a label-free, prior-free alerting mechanism that automatically detects whether LLM-based adjudicators systematically assign scores below a predefined threshold. Experiments demonstrate its effectiveness in identifying single or multiple anomalous scoring sources, achieving both theoretical rigor—via formal logic and optimization—and practical robustness in unsupervised evaluation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

If two experts disagree on a test, we may conclude both cannot be 100 per cent correct. But if they completely agree, no possible evaluation can be excluded. This asymmetry in the utility of agreements versus disagreements is explored here by formalizing a logic of unsupervised evaluation for classifiers. Its core problem is computing the set of group evaluations that are logically consistent with how we observe them agreeing and disagreeing in their decisions. Statistical summaries of their aligned decisions are inputs into a Linear Programming problem in the integer space of possible correct or incorrect responses given true labels. Obvious logical constraints, such as, the number of correct responses cannot exceed the number of observed responses, are inequalities. But in addition, there are axioms, universally applicable linear equalities that apply to all finite tests. The practical and immediate utility of this approach to unsupervised evaluation using only logical consistency is demonstrated by building no-knowledge alarms that can detect when one or more LLMs-as-Judges are violating a minimum grading threshold specified by the user.

Problem

• Formalizing logic for unsupervised classifier evaluation consistency
• Computing group evaluations consistent with expert agreement patterns
• Developing no-knowledge alarms for LLM grading threshold violations

Innovation

• Formalizing logic of unsupervised classifier evaluation
• Using linear programming with logical consistency constraints
• Building no-knowledge alarms for LLM grading violations