When Do Generalized Permutation Tests Achieve Optimal Power? A Dispersion Characterization

2026-05-30

AI Summary

This study investigates the conditions under which generalized permutation tests achieve optimal power under non-uniform randomization distributions. Focusing on Monte Carlo permutation tests based on the mean difference statistic, the work introduces individual- and pair-level dispersion measures to quantify deviations from complete randomization. Theoretical analysis shows that when both dispersion measures asymptotically vanish, the permutation distribution converges to a Gaussian limit, yielding stable critical values and attaining Pitman local asymptotic optimality; otherwise, optimality cannot be guaranteed. The key contribution lies in demonstrating that non-uniform randomization schemes can exploit structural heterogeneity in the data to outperform uniform designs, thereby establishing a rigorous connection among dispersion metrics, distributional convergence, and testing power.

Abstract

We study generalized Monte Carlo permutation tests under a non-uniform distribution on permutations. Focusing on the difference-in-means statistic, we introduce two scalar dispersion measures that quantify departures from complete randomization at the individual and pairwise levels. We show that if both dispersions vanish asymptotically, then the conditional permutation distribution converges to its Gaussian benchmark, the critical value stabilizes, and the test attains optimal Pitman local power. Conversely, if these dispersions fail to vanish, the permutation distribution does not self-average, the critical value need not stabilize, and optimal local power cannot in general be guaranteed. We further show that beyond the standard Pitman local model, suitably chosen non-uniform permutation distributions can strictly dominate the uniform distribution by exploiting nuisance structure in the data.

Problem

• permutation tests
• optimal power
• non-uniform permutations
• dispersion measures
• Pitman local power

Innovation

• generalized permutation tests
• dispersion characterization
• optimal Pitman power
• non-uniform permutations
• self-averaging