Differentially private testing for relevant dependencies in high dimensions

2025-11-21

AI Summary

This paper addresses differentially private detection of variable dependencies in high-dimensional data, focusing on testing whether the absolute values of all pairwise Kendall’s τ coefficients are bounded by a given threshold—a composite null hypothesis—rather than conventional pairwise independence testing. We propose a novel adaptive privacy-preserving test: it employs the Laplace mechanism to achieve ε-differential privacy and introduces a bootstrap-based statistical inference framework operating on perturbed samples. Theoretically, the method controls Type-I error and achieves asymptotic power under sparse high-dimensional regimes. Empirically, it maintains high detection power even under stringent privacy constraints (ε ≤ 1) and small sample sizes. Applied to real clinical data, the method uncovers bounded weak dependence structures among medical variables, offering a new paradigm for high-dimensional dependency modeling in privacy-sensitive settings.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Relational Probabilistic Models
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We investigate the problem of detecting dependencies between the components of a high-dimensional vector. Our approach advances the existing literature in two important respects. First, we consider the problem under privacy constraints. Second, instead of testing whether the coordinates are pairwise independent, we are interested in determining whether certain pairwise associations between the components (such as all pairwise Kendall's $τ$ coefficients) do not exceed a given threshold in absolute value. Considering hypotheses of this form is motivated by the observation that in the high-dimensional regime, it is rare and perhaps impossible to have a null hypothesis that can be modeled exactly by assuming that all pairwise associations are precisely equal to zero. The formulation of the null hypothesis as a composite hypothesis makes the problem of constructing tests already non-standard in the non-private setting. Additionally, under privacy constraints, state of the art procedures rely on permutation approaches that are rendered invalid under a composite null. We propose a novel bootstrap based methodology that is especially powerful in sparse settings, develop theoretical guarantees under mild assumptions and show that the proposed method enjoys good finite sample properties even in the high privacy regime. Additionally, we present applications in medical data that showcase the applicability of our methodology.

Problem

• Detecting dependencies in high-dimensional vectors under privacy constraints
• Testing if pairwise associations exceed given thresholds rather than independence
• Developing bootstrap methods for composite null hypotheses with privacy guarantees

Innovation

• Bootstrap-based dependency testing under privacy constraints
• Composite null hypothesis for high-dimensional pairwise associations
• Sparse setting optimization with theoretical guarantees