On the universal calibration of Pareto-type linear combination tests

2025-09-15

AI Summary

This paper addresses the inflation of Type I error in multiple p-value combination tests caused by tail dependence. We propose a universal calibration method for Pareto-type linear combination tests. Leveraging multivariate regular variation theory and angular measure analysis, we model the tail dependence structure of homogeneous heavy-tailed p-values and conduct asymptotic analysis over homogeneous heavy-tailed function families. Theoretically, we prove that the proposed test is universally calibrated under the global null hypothesis: its asymptotic significance level equals the nominal level exactly, regardless of tail dependence strength—including perfect tail dependence. In contrast, the Cauchy test is honest but conservative, while the Tippett test is only calibratable under tail independence. This work establishes, for the first time, the unique universal calibration property of Pareto-type combinations in heavy-tailed, dependent settings, providing a robust, non-conservative benchmark for high-dimensional multiple testing.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Game Theory and Economic Paradigms: Behavioral Game Theory

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

It is often of interest to test a global null hypothesis using multiple, possibly dependent, $p$-values by combining their strengths while controlling the Type I error. Recently, several heavy-tailed combinations tests, such as the harmonic mean test and the Cauchy combination test, have been proposed: they map $p$-values into heavy-tailed random variables before combining them in some fashion into a single test statistic. The resulting tests, which are calibrated under the assumption of independence of the $p$-values, have shown to be rather robust to dependence. The complete understanding of the calibration properties of the resulting combination tests of dependent and possibly tail-dependent $p$-values has remained an important open problem in the area. In this work, we show that the powerful framework of multivariate regular variation (MRV) offers a nearly complete solution to this problem. We first show that the precise asymptotic calibration properties of a large class of homogeneous combination tests can be expressed in terms of the angular measure -- a characteristic of the asymptotic tail-dependence under MRV. Consequently, we show that under MRV, the Pareto-type linear combination tests, which are equivalent to the harmonic mean test, are universally calibrated regardless of the tail-dependence structure of the underlying $p$-values. In contrast, the popular Cauchy combination test is shown to be universally honest but often conservative; the Tippet combination test, while being honest, is calibrated if and only if the underlying $p$-values are tail-independent. One of our major findings is that the Pareto-type linear combination tests are the only universally calibrated ones among the large family of possibly non-linear homogeneous heavy-tailed combination tests.

Problem

• Universal calibration of combination tests for dependent p-values
• Analyzing asymptotic tail-dependence using multivariate regular variation
• Identifying Pareto-type tests as only universally calibrated method

Innovation

• Universal calibration using multivariate regular variation framework
• Pareto-type linear tests equivalent to harmonic mean method
• Asymptotic calibration via angular measure characterization