Power Studies For Two-sample Methods For Multivariate Data

2025-07-22

AI Summary

Nonparametric multivariate two-sample tests suffer from unstable statistical power and lack of universally robust solutions. Method: We systematically evaluated over ten distance-based, kernel-based, and permutation-based nonparametric methods via large-scale simulations across continuous/discrete multivariate data and diverse null-alternative hypothesis configurations. We introduced the “minimal complete method set” strategy to identify a compact, complementary subset of 3–5 highly robust and broadly applicable tests. Contribution/Results: This curated ensemble achieves ≥90% relative power across distributional shifts—including location, scale, and shape differences—substantially outperforming any single method. Implemented in the R package MD2sample, our framework ensures reproducible analysis and delivers a practical, theoretically grounded guide for multivariate two-sample testing that balances rigor with implementability.

Technology Category

• Machine Learning: Ensemble Methods
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

We present the results of a large number of simulation studies regarding the power of various non-parametric two-sample tests for multivariate data. This includes both continuous and discrete data. In general no single method can be relied upon to provide good power, any one method may be quite good for some combination of null hypothesis and alternative and may fail badly for another. Based on the results of these studies we propose a fairly small number of methods chosen such that for any of the case studies included here at least one of the methods has good power. The studies were carried out using the R package MD2sample, available from CRAN.

Problem

• Evaluate power of non-parametric two-sample multivariate tests
• Compare methods for continuous and discrete data scenarios
• Propose optimal test selections for diverse hypothesis cases

Innovation

• Simulation studies on multivariate two-sample tests
• Non-parametric methods for diverse data types
• Optimal test selection via R package MD2sample