Conformal Prediction under Partial Verification

2026-10-07

AI Summary

This study addresses the high cost of label verification in conformal prediction by proposing a partial verification method based on calibration certificates. The approach introduces the first partial verification mechanism capable of returning prediction sets identical to those obtained through full verification, along with a coordination strategy. By integrating an ordered candidate checking algorithm and determining thresholds via calibration certificates, the method optimizes the verification process. Experimental results demonstrate that the proposed approach reduces verification costs by 15% to 82% across multiple scenarios while strictly guaranteeing that the resulting prediction sets remain fully consistent with those produced by full verification. Consequently, this work successfully unifies computational efficiency with rigorous theoretical guarantees in conformal prediction.

Abstract

Conformal prediction provides prediction sets with finite-sample guarantees, but the label verification required for calibration can be expensive. We develop a partial verification method that returns exactly the same prediction sets as complete verification. We characterize calibration certificates, the verified information sufficient to determine the conformal threshold, and design a procedure that coordinates verification across calibration examples. For finite thresholds at high coverage, its verification cost is less than twice the minimum certificate cost when candidates are checked in order. Across retrieval, mathematical solutions, and configuration evaluation, it reduces verification cost by 15-82% compared with verifying calibration examples one at a time, while producing identical prediction sets.

Problem

• Conformal Prediction
• Partial Verification
• Calibration Cost
• Label Verification

Innovation

• Conformal Prediction
• Partial Verification
• Calibration Certificates
• Verification Cost Reduction
• Finite-Sample Guarantees