likelihood ratio test

Statistical hypothesis testing based on comparing likelihoods of nested models via their ratio to assess effects (e.g., skewness) or discriminability between distributions; involves computing the test statistic, characterizing its null distribution, and deriving sample‑size requirements for reliable inference.

likelihoodratiotest

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the well-known issue that conventional likelihood ratio tests exhibit substantial size distortions under small samples when testing composite null hypotheses involving equality and inequality constraints, unions of multiple regions, or nuisance parameters, particularly due to violations of the regularity condition requiring a boundary-free manifold. To overcome these limitations, the authors propose a finite-sample testing procedure that conducts pointwise simple hypothesis tests across the entire null parameter space and applies a significance-level inflation correction to achieve overall size control. This approach accommodates generalized composite null structures beyond the scope of traditional methods. Numerical experiments demonstrate that the proposed test maintains actual size extremely close to the nominal level, with negligible distortion, in both small- and large-sample settings.

composite null hypothesesfinite-sample testinglikelihood ratio test

This paper addresses optimal hypothesis testing under parametrically dependent support sets—i.e., nonregular models. For both one-sided and two-sided testing problems, we propose asymptotically uniformly most powerful (AUMP) tests based on the likelihood ratio process. We introduce, for the first time, a randomized one-sided test achieving exact asymptotic level control; for two-sided testing, the proposed test remains AUMP under standard unbiasedness constraints. A key innovation is the incorporation of a tuning constant that eliminates discontinuities in the limiting likelihood ratio process. Rigorous theoretical analysis establishes the asymptotic optimality of both tests. Monte Carlo simulations demonstrate that the proposed procedures attain substantially higher finite-sample power than existing methods, particularly in small- to moderate-sized samples.

Addresses parameter-dependent support in one-sided and two-sided testingConstructs asymptotically uniformly most powerful tests and confidence setsDevelops optimal hypothesis tests for nonregular econometric models

Modelling Sampling Distributions of Test Statistics with Autograd

May 03, 2024
AA
A. A. Kadhim
🏛️ Florida State University

This work addresses the problem of exact calibration of conditional confidence sets in simulation-based inference—specifically, accurately modeling the conditional sampling distribution (or p-value function) of a scalar test statistic given compressed observations, to guarantee strict conditional coverage. To this end, we propose an automatic-differentiation-driven neural network method that directly learns the one-dimensional conditional cumulative distribution function (CDF) of the test statistic; its derivative, computed via backpropagation, serves as a differentiable approximation of the conditional probability density function (PDF), unifying CDF modeling and PDF estimation for the first time. Leveraging uncertainty quantification techniques—including Monte Carlo Dropout, ensemble averaging, and quantile regression—we substantially improve coverage reliability. Experiments demonstrate that our approach strictly achieves the target conditional coverage across multiple tasks, while significantly reducing modeling complexity and sampling overhead compared to state-of-the-art density-ratio estimation baselines.

Comparing neural network approach to probability density-ratio methodModeling sampling distributions of test statistics using autogradQuantifying predictive uncertainty in conditional 1-dimensional distributions

This study addresses the theoretical gap between classical hypothesis testing with fixed significance levels and Bayesian methods, particularly in light of the Lindley paradox. By leveraging moderate deviation theory, the authors develop a unified Bayesian framework for hypothesis testing. Through Bayesian risk analysis and asymptotic expansions, they show that the optimal test threshold operates on the scale of √(log n / n), naturally yielding Jeffreys’ threshold, the BIC penalty term, and the Chernoff–Stein error exponent. This framework not only resolves the Lindley paradox but also extends Rubin’s (1965) program to modern settings such as high-dimensional sparse inference, goodness-of-fit testing, and model selection. Moreover, it establishes the superiority of Bayesian procedures over classical Neyman–Pearson tests in terms of statistical risk.

Bayes riskBayesian hypothesis testingLindley paradox

Assessing Inference Methods

Dec 18, 2019
BF
Bruno Ferman
🏛️ Sao Paulo School of Economics - FGV

This study addresses the uncontrolled false positive rates and misleading inferences arising from commonly used simulation methods in shift-share designs. We systematically evaluate prevailing inferential approaches in empirical research through a suite of multilevel simulation experiments. By comparing Monte Carlo analysis with counterfactual data-generating mechanisms, we uncover non-monotonic trade-offs among fidelity, sensitivity, and risk of misdirection across simulation designs. We propose a novel “progressive-fidelity simulation framework,” demonstrating that low-fidelity simulations suffice to expose fundamental inferential flaws, whereas high-fidelity simulations detect subtle, previously overlooked biases—substantially improving detection power. The framework balances interpretability and computational efficiency, offering a reproducible and scalable paradigm for assessing the robustness of causal inference methods.

Analyzing trade-offs in simulation-based inference assessmentsEvaluating reliability of inference methods for false-positive controlProposing alternatives to misleading shift-share design evaluations

Latest Papers

What's happening recently
View more

Traditional hypothesis testing faces validity challenges in modern settings involving abstract null hypotheses, post hoc selection of inference targets, and weak distributional assumptions. This work proposes a novel approach that circumvents the need for pre-specified selection mechanisms or strong distributional assumptions. The method decomposes the data into two parts via symmetric noise injection and, under the null hypothesis, conditionally orthogonalizes one part with respect to the other; validity is then assessed by testing whether this orthogonality holds. Grounded in the theory of symmetric shift families, the procedure demonstrates broad applicability and flexibility across a variety of complex null hypotheses and post-selection scenarios, substantially expanding the scope of valid hypothesis testing.

data splittinghypothesis testingnull hypothesis

This study addresses the limitation of conventional global goodness-of-fit tests in multivariate settings, which often fail to pinpoint localized model misspecifications. To overcome this, the authors propose a local calibration test based on adaptive partitioning via Beta-trees. Departing from single-statistic global frameworks, the method evaluates whether predicted probabilities fall within finite-sample confidence intervals across data-driven subregions, enabling precise identification and visualization of model inadequacies. By leveraging k-means clustering to generate null distributions and constructing rigorous confidence intervals, the approach effectively detects local deviations in both simulated and real-world datasets, demonstrating superior performance in tasks such as selecting the number of components in mixture models.

Beta-treesgoodness-of-fitlocal deviations

This work addresses the problem of multiple hypothesis testing for edge distributions across multiple data streams. It proposes a sequential testing procedure that, for the first time, systematically incorporates arbitrary forms of prior information about the configuration of true and false hypotheses—such as known values or lower bounds on the number of active streams under each hypothesis, or mutual exclusivity constraints—while rigorously controlling the familywise error rate. By integrating sequential analysis with a search strategy over minimal alternative hypothesis configurations, the method achieves asymptotic optimality in terms of expected sample size among all valid procedures, without compromising reliability. Theoretical analysis establishes its computational efficiency and asymptotic optimality, and numerical experiments further demonstrate its substantial advantages in both testing efficiency and accuracy.

familywise errorhypothesis configurationmultiple hypotheses

This study addresses the challenge of testing equality of conditional distributions in settings with high-dimensional covariates and multivariate responses. The authors propose a cross-generative alignment approach that circumvents direct estimation of conditional density ratios by training two conditional generators and cross-generating responses at each other’s covariate values, thereby enabling direct comparison between generated and observed samples. A test statistic is constructed via an indexed empirical process in a reproducing kernel Hilbert space (RKHS), with inference carried out using a multiplier bootstrap. The method is theoretically shown to be consistent under both the null and alternative hypotheses, to possess a well-characterized limiting distribution, and to admit valid bootstrap approximation. Empirical results demonstrate its superior performance in high-dimensional regimes over existing methods, exhibiting both double robustness and strong adaptability to limited covariate overlap.

conditional discrepancyconditional distribution equalitygenerative models

This study addresses the problem of testing conditional independence between random variables \( X \) and \( Y \) given a confounding variable \( Z \). It proposes a local permutation test based on data-adaptive binning—such as equal-count binning—where permutations of \( X \) and \( Y \) are performed within each subregion defined by \( Z \). The method provides, for the first time, finite-sample Type I error control guarantees for arbitrary test statistics. Under linear confounding models, it achieves power comparable to that of the oracle likelihood ratio test. Theoretical analysis shows that a constant bin size suffices to attain performance on par with increasing bin sizes, and numerical experiments confirm the method’s statistical efficiency and practical utility.

conditional independenceconfounderdata-adaptive binning

Hot Scholars

AR

Aaditya Ramdas

Associate Professor (with tenure), Carnegie Mellon University
Machine LearningStatistics
AR

Anders Rahbek

Professor, University of Copenhagen
EconometricsTime seriesGARCHCointegration
JQ

Jing Qin

University of Southern Denmark
MathematicsStatistics
CA

Chunrong Ai

The Chinese University of Hong Kong, Shenzhen
econometrics