Amortized Inference for Sampling Distributions Where the Bootstrap Fails

2026-07-18

AI Summary

This work addresses the failure of the classical bootstrap in settings such as bounded-support maxima, infinite-variance means, extreme quantiles, and tail index estimation, where existing corrections either rely on unknown parameters or exhibit poor small-sample stability. The authors propose the first amortized inference framework tailored to such bootstrap-failure scenarios: a neural network trained on a prior over distribution families learns to score the root statistic \(T_n - T(F)\) via pinball loss, enabling a single forward pass on \(n=200\) observations to output an entire sampling distribution and corresponding confidence intervals. Requiring no problem-specific tuning, a single universal network handles diverse statistical tasks, achieving nominal coverage near 95% across four canonical failure cases with Wasserstein distances superior to all viable classical alternatives; on real market data, it attains an average coverage of 0.87, substantially outperforming the bootstrap’s 0.73.

Abstract

Efron's bootstrap is the default tool for estimating the sampling distribution of a statistic, yet it is provably inconsistent for maxima of bounded-support distributions, means under infinite variance, extreme quantiles, and tail-index estimators. The classical remedies, the m-out-of-n bootstrap and subsampling, require rate corrections that depend on unknown parameters and behave erratically at realistic sample sizes. We propose an amortized alternative: a neural network is trained on simulated datasets drawn from a prior over a distribution family, using single independent draws of the root T_n - T(F) scored by the pinball loss, a proper scoring rule whose population minimizer is the posterior-predictive law of the root. At test time, a single forward pass maps one dataset of n = 200 observations to its full sampling-distribution estimate, from which confidence intervals follow directly. On four canonical bootstrap-failure problems (bounded-support maximum, alpha-stable mean, Pareto tail index, and 99% value-at-risk under tempered stable returns), the method attains nominal 95% coverage, beats every feasible classical method in Wasserstein distance to the true sampling distribution, and captures over 97% of the achievable improvement where the exact Bayes-optimal answer is computable. For the value-at-risk problem no distribution-free method can reach nominal coverage at all; the learned method attains 94.7%. A single universal network with a statistic token matches all four specialists, and on real daily market returns the unchanged model averages 0.87 coverage against 0.73 for the bootstrap, as predicted by our out-of-family analysis.

Problem

• bootstrap failure
• sampling distribution
• extreme quantiles
• tail-index estimation
• infinite variance

Innovation

• amortized inference
• bootstrap failure
• neural network
• pinball loss
• sampling distribution