Stratified Bootstrap Test Package

2025-12-16

AI Summary

This study addresses the stability assessment of group-specific ranking patterns and nonparametric inference on population-mean ordinal relationships in multivariate survey/scoring data. We propose the first hierarchical bootstrap framework for ordinal hypothesis testing, which approximates the null distribution without distributional assumptions. We introduce the *non-containment index*—a robust, interpretable measure quantifying ranking stability across groups—and leverage it for outlier response detection and significance testing of inter-group ranking differences. The method integrates hierarchical resampling, nonparametric stability analysis, and resampling-based ordinal inference, unifying descriptive and inferential capabilities. Evaluated in AI fairness auditing and questionnaire analysis, it demonstrates high sensitivity and reliability. Our approach establishes a novel, assumption-free, robust, and interpretable statistical paradigm for ordinal data analysis.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

The Stratified Bootstrap Test (SBT) provides a nonparametric, resampling-based framework for assessing the stability of group-specific ranking patterns in multivariate survey or rating data. By repeatedly resampling observations and examining whether a group's top-ranked items remain among the highest-scoring categories across bootstrap samples, SBT quantifies ranking robustness through a non-containment index. In parallel, the stratified bootstrap test extends this framework to formal statistical inference by testing ordering hypotheses among population means. Through resampling within groups, the method approximates the null distribution of ranking-based test statistics without relying on distributional assumptions. Together, these techniques enable both descriptive and inferential evaluation of ranking consistency, detection of aberrant or adversarial response patterns, and rigorous comparison of groups in applications such as survey analysis, item response assessment, and fairness auditing in AI systems.

Problem

• Assesses stability of group-specific ranking patterns in multivariate data
• Quantifies ranking robustness using a non-containment index via resampling
• Enables descriptive and inferential evaluation of ranking consistency across groups

Innovation

• Nonparametric resampling framework for ranking stability
• Stratified bootstrap approximates null distribution without assumptions
• Enables descriptive and inferential evaluation of ranking consistency