Multigroup Fairness and Omniprediction: Separations and Equivalences

2026-10-05

AI Summary

This study investigates whether omniscient prediction necessarily depends on multiple fairness concepts and examines their logical relationship. Employing theoretical computer science tools—including learning guarantees, distribution indistinguishability, and fairness metrics—the authors conduct formal derivations to address this question. The work makes two primary contributions: it is the first to demonstrate that standard omniscient prediction does not imply expected accuracy, thereby requiring no support from multi-group fairness notions; and it rigorously establishes the equivalence between loss indistinguishability and calibrated multi-accuracy. By delineating the boundaries of separation and equivalence among distinct prediction paradigms and fairness metrics, this research provides a novel theoretical foundation for algorithm design in fair machine learning.

Abstract

Omniprediction is a learning guarantee which requires a single predictor to be competitive relative to the best hypothesis from a benchmark class for any loss chosen from a family of loss functions. Loss Outcome Indistinguishability (loss OI for short) is a stronger notion that implies omniprediction. It requires the predicted distribution on labels to be indistinguishable from the true distribution to tests that depend on the loss functions and the benchmark class. Multiaccuracy and multicalibration are multigroup fairness notions that generalize classical notions of calibration and accuracy in expectation. Most known learning algorithms for omniprediction (both for the standard notion and for strengthenings like loss OI) rely on some version of these multigroup fairness notions, or on an intermediate notion called calibrated multiaccuracy. We ask if this is necessary: Does omniprediction require some form of multigroup fairness? We show that the answer is no for (plain) omniprediction, and yes for loss OI. First, a sequence of works shows that multicalibration or calibrated multiaccuracy imply omniprediction. We rule out even a weak converse, by showing that omniprediction for proper losses does not imply even accuracy in expectation, a much weaker notion than any of calibration, multiaccuracy, or multicalibration. Second, prior work showed how to achieve loss OI from a combination of calibration and multiaccuracy. We show a converse: loss OI is equivalent to a form of calibrated multiaccuracy.

Problem

• Omniprediction
• Loss Outcome Indistinguishability
• Multigroup Fairness
• Multiaccuracy
• Multicalibration

Innovation

• Omniprediction
• Loss Outcome Indistinguishability
• Multigroup Fairness
• Calibrated Multiaccuracy
• Multiaccuracy