Meager Success: A Theory of the Unlearnable for Hypothesis Testing

2026-07-03

AI Summary

When pointwise consistency is unattainable, what is the strongest achievable convergence criterion in hypothesis testing? This work proposes a topological framework of weakened consistency, requiring estimators to converge to the true parameter on a “large” set of probability measures—specifically, a dense set—rather than everywhere. By leveraging topological notions such as comeager sets and dense subsets, the paper establishes a realizability theorem that is strictly weaker than pointwise consistency and demonstrates that, under finite precision, convergence on a dense set necessarily entails inconsistency on a comeager set. This result exposes fundamental limitations inherent in distribution-free hypothesis testing (e.g., conditional independence tests) and offers a novel topological perspective on statistical learnability.

Abstract

When the standard of pointwise consistency for statistical inference -- convergence to the truth in every possible state of the world -- is provably unachievable, the usual responses are to change the inferential target or to strengthen background assumptions. This paper pursues a third: hold the inference problem fixed and identify the highest standard that remains achievable. I define a hierarchy of standards weaker than pointwise consistency, cast in topological terms, requiring convergence to the truth not everywhere but on a ``large'' set of probability measures. The main result is an impossibility theorem: for finite-precision tests, converging to the truth densely within each hypothesis already forces inconsistency on a comeager -- ``topologically almost all'' -- set of measures, whenever the two hypotheses are dense in their union. Distribution-free testing of conditional independence is one such case. Two further theorems characterize, in purely topological terms, exactly when each weaker standard is achievable, complementing Boeken et al.'s (2026) analysis of pointwise consistency.

Problem

• unlearnable
• pointwise consistency
• hypothesis testing
• topological characterization
• conditional independence

Innovation

• unlearnability
• pointwise consistency
• topological impossibility
• hypothesis testing
• comeager sets