proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference

2026-07-27

AI Summary

Although proxy metrics are commonly employed to substitute for hard-to-observe primary outcomes, their systematic biases often undermine inference validity and distort confidence intervals. This work proposes the *proxymate* framework—a systematic, modular four-layer diagnostic and correction system encompassing representativeness, unit, estimation, and domain levels—that maps specific failure modes to targeted remediation strategies. Implemented through hierarchical diagnostics, bias-correction algorithms, and an open-source Python toolkit, the framework supports diverse applications including experimentation, monitoring, and prevalence estimation. Validated across thousands of Meta experiments and multiple product lines on millions of proxy–primary outcome pairs, *proxymate* significantly enhances inference reliability and accelerates decision-making.

Abstract

Proxy outcomes (such as short-term behavioral signals, model predictions, or surrogate endpoints) are frequently used in place of primary outcomes that are too slow to mature, rare, or challenging to measure directly. But valid inference on a proxy does not guarantee valid inference on the primary estimate as proxy-based estimates can be systematically biased in ways that are difficult to predict, leading to improperly calibrated confidence intervals. We present proxymate, a framework and open-source Python package for proxy validation and adjustment. proxymate organizes into four levels: The Representativity Level (population validity), the Unit Level (measurement quality), the Estimate Level (decision validity), and the Domain Level (cross-domain transportability). Within each level, proxymate provides diagnostic checks, and targeted adjustment strategies that map specific failures to appropriate corrections. At Meta, proxymate has been adopted by many different use cases, spanning experimentation, prevalence estimation, and monitoring use cases, all facing different proxy challenges (limited human review time, long maturation window of outcomes, low detectability) and showcasing the modularity of the framework. Across all products, proxymate assessed and corrected millions of proxy, primary unit comparisons. It has facilitated launches across multiple work streams including enabling quick decision making on thousands of experiments.

Problem

• proxy outcomes
• systematic bias
• valid inference
• confidence interval calibration
• surrogate endpoints

Innovation

• proxy validation
• bias correction
• reliable inference
• modular framework
• surrogate endpoints