significance testing

This paper addresses the complexity and high pedagogical/practical barriers associated with conventional uncertainty quantification methods—such as standard errors, confidence intervals, and hypothesis tests—in statistical inference. To evaluate the potential of nonparametric bootstrap as a unified alternative, we conduct a large-scale simulation study rigorously comparing single bootstrap, double bootstrap, and classical methods across multiple dimensions: sample size, confidence level, data-generating mechanisms, and statistical functionals. Results demonstrate that the double bootstrap consistently achieves superior coverage accuracy, stability, and robustness—particularly under small-sample and non-normal conditions—outperforming both classical approaches and the single bootstrap. We thus establish the double bootstrap as a principled, parsimonious, and high-performance paradigm for uncertainty quantification, providing both theoretical justification and empirical evidence to support its adoption in statistical education and applied practice.

This work proposes a nonparametric method to assess the statistical significance of signal features—such as peaks and plateaus—in data and to detect multimodal structures in inter-event spacing distributions. The approach leverages run theory, employing a Markov chain recursion to precisely characterize the distribution of the longest runs. It integrates permutation testing with a bootstrap procedure tailored for continuous data, enabling a unified evaluation of both high- and low-intensity signal features. The key innovation lies in the first principled synthesis of run-length analysis, permutation tests, and continuous-data bootstrapping, which collectively facilitate accurate detection and localization of multimodal patterns without requiring parametric distributional assumptions, thereby effectively identifying salient morphological features in complex datasets.

This paper addresses the problem of testing parameter homogeneity across multiple high-dimensional populations (with large $k$) under small per-group sample sizes. We propose an asymptotically distribution-free nonparametric test that relaxes the classical assumptions of fixed $k$ and large overall sample size. The test statistic is constructed via asymptotic distribution theory and enhanced by linear bootstrap calibration to improve finite-sample accuracy. We establish its asymptotic exactness under the null hypothesis and consistency under alternatives. Simulation studies demonstrate superior finite-sample performance compared to existing methods, and real-data analyses confirm its robustness and practical utility in high-dimensional, multi-group settings. The key contribution is the first nonparametric homogeneity test framework specifically designed for large-$k$, small-$n$ scenarios—offering both rigorous theoretical guarantees (asymptotic validity and consistency) and strong empirical performance.

This study addresses the stability assessment of group-specific ranking patterns and nonparametric inference on population-mean ordinal relationships in multivariate survey/scoring data. We propose the first hierarchical bootstrap framework for ordinal hypothesis testing, which approximates the null distribution without distributional assumptions. We introduce the *non-containment index*—a robust, interpretable measure quantifying ranking stability across groups—and leverage it for outlier response detection and significance testing of inter-group ranking differences. The method integrates hierarchical resampling, nonparametric stability analysis, and resampling-based ordinal inference, unifying descriptive and inferential capabilities. Evaluated in AI fairness auditing and questionnaire analysis, it demonstrates high sensitivity and reliability. Our approach establishes a novel, assumption-free, robust, and interpretable statistical paradigm for ordinal data analysis.

This paper addresses the lack of generality and theoretical foundations in existing bootstrap hypothesis testing frameworks. We propose a unified bootstrap testing framework that accommodates both null-distribution-based resampling and diverse nonstandard bootstrap schemes. We first systematically characterize the exchangeability condition and statistical functional construction criteria, prove the local asymptotic equivalence of different resampling schemes in terms of statistical power, and identify the intrinsic mechanism behind the failure of the naive bootstrap. Leveraging empirical process theory and weak convergence analysis, we rigorously establish the asymptotic exactness and consistency of the test under fixed alternatives. An accompanying open-source R package, *BootstrapTests*, validates the theoretical properties in independence testing, linear regression coefficient testing, and copula model goodness-of-fit testing. Finite-sample simulations demonstrate that the proposed method significantly improves statistical power.

Traditional chi-square tests can only assess exact independence between variables in contingency tables and are ill-suited for capturing the approximate independence commonly encountered in practice. This work proposes an equivalence testing framework tailored for two-dimensional contingency tables, which employs a boundary-point estimator combined with asymptotic critical values and Bootstrap resampling to enhance performance in small samples while maintaining statistical efficiency. The method is the first to be systematically applicable across contingency tables of varying dimensions, offering both theoretical rigor and computational feasibility. Extensive simulations demonstrate its superior performance across diverse table sizes, and its practical utility is further corroborated through real-data applications. The accompanying implementation code has been made publicly available.

This study addresses the challenge of testing equality of conditional distributions in settings with high-dimensional covariates and multivariate responses. The authors propose a cross-generative alignment approach that circumvents direct estimation of conditional density ratios by training two conditional generators and cross-generating responses at each other’s covariate values, thereby enabling direct comparison between generated and observed samples. A test statistic is constructed via an indexed empirical process in a reproducing kernel Hilbert space (RKHS), with inference carried out using a multiplier bootstrap. The method is theoretically shown to be consistent under both the null and alternative hypotheses, to possess a well-characterized limiting distribution, and to admit valid bootstrap approximation. Empirical results demonstrate its superior performance in high-dimensional regimes over existing methods, exhibiting both double robustness and strong adaptability to limited covariate overlap.

Traditional goodness-of-fit tests struggle to distinguish between “no significant difference” and “practical equivalence,” as failure to reject the null hypothesis may merely reflect insufficient test power. This work proposes the first kernel-based framework for full-distribution equivalence testing, leveraging Kernel Stein Discrepancy (KSD) and Maximum Mean Discrepancy (MMD) to quantify the distance between distributions while incorporating a prespecified minimum equivalence margin. By employing asymptotic normal approximations and bootstrap procedures to compute critical values, the method overcomes the limitations of existing equivalence tests, which are typically confined to parametric models or specific moments. Numerical experiments demonstrate that the proposed approach reliably assesses whether two distributions are equivalent within the specified margin while effectively controlling both Type I and Type II error rates.

This study addresses ongoing controversies surrounding null hypothesis significance testing—particularly the mechanical reliance on fixed p-value thresholds, rigid treatment of null and alternative hypotheses, and calls for abandoning the method altogether. By systematically evaluating recent reform proposals and integrating frequentist and Bayesian perspectives, the work re-examines the theoretical foundations and practical logic of significance testing. It clarifies the strengths and limitations of various reform approaches, demonstrates that sampling distributions can be constructed without pre-specifying thresholds or alternative hypotheses, and synthesizes techniques from hypothesis testing, Bayesian decision theory, confidence intervals, and sampling distribution analysis to expose key misinterpretations in current practice. The study argues that significance testing retains value for scientific inference when applied with greater caution and flexibility, rather than being discarded outright.

Traditional significance testing often fosters dichotomous thinking, misinterpretation, and irreproducibility, thereby undermining robust scientific inference. This work proposes a unified evidence- and decision-oriented inferential framework that integrates advances in statistical inference and open science practices from 2016 to 2026. Moving beyond sole reliance on p-values, the framework incorporates compatibility interpretations, S-values, smallest effect sizes of interest (SESOI) equivalence testing, Bayesian workflows, and e-value sequential inference. It is further embedded within open science infrastructure—including preregistration, registered reports, multiverse analyses, and adherence to PRISMA 2020 and CONSORT 2025 reporting standards. By synergistically advancing methodological innovation and institutional reform, this open science inference system substantially enhances research transparency, reproducibility, and the quality of scientific decision-making.