Variance Estimation with Dependence and Heterogeneous Means

2026-03-11

AI Summary

This study addresses the problem that standard variance estimators systematically underestimate the true variance—and consequently lead to over-rejection in hypothesis tests—when random vectors exhibit heterogeneous means and possess either clustered or weakly dependent structures. The paper proposes a novel conservative variance estimator for sums of random vectors in a triangular array setting, which is robust to mean heterogeneity and accommodates two-way clustering or weak dependence. Leveraging asymptotic theory and robust construction techniques, the proposed method maintains computational feasibility while effectively controlling test size, thereby avoiding the excessive rejection rates associated with conventional approaches and ensuring valid statistical inference.

Technology Category

Reasoning under Uncertainty: Stochastic OptimizationMachine Learning: Ensemble MethodsIntelligent Robots: State Estimation

Application Category

Security and Privacy: Large-scale security measurementsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methodsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper considers the problem of estimating the variance of a sum of a triangular array of random vectors with heterogeneous means. When random vectors exhibit two-way cluster dependence or weak dependence, standard variance estimators designed under homogeneous means can underestimate the true variance, which results in subsequent tests being oversized. To restore validity, this paper proposes a simple conservative variance estimator robust to heterogeneous means and shows its asymptotic validity.

Problem

• variance estimation
• heterogeneous means
• cluster dependence
• weak dependence
• triangular array

Innovation

• variance estimation
• heterogeneous means
• cluster dependence
• asymptotic validity
• conservative estimator