flow-based density estimation

This work proposes a novel approach that integrates normalizing flows with stratified sampling to estimate expectations without relying on restrictive (semi-)parametric distributional assumptions, such as Gaussian or Gaussian mixture models, which can introduce substantial bias when misspecified. By leveraging the expressive power of neural networks, the method flexibly captures complex, unknown data distributions, thereby overcoming the limitations of traditional parametric frameworks. Empirical evaluations demonstrate that the proposed estimator significantly reduces Monte Carlo uncertainty in high-dimensional settings—specifically in 30- and 128-dimensional problems—and achieves marked improvements in both accuracy and stability compared to conventional Monte Carlo estimators and Gaussian mixture model-based approaches.

This work addresses the challenging problem of density estimation for complex target distributions that combine an analytically tractable component with a non-analytic component accessible only through samples from a simulator or dataset, where direct sampling or closed-form evaluation is infeasible. The authors propose a two-stage normalizing flow framework: the first stage learns the density of the non-analytic component from samples, and the second stage integrates this estimate with the analytic term to reconstruct the full target distribution, enabling efficient sampling and density evaluation. This approach uniquely unifies heterogeneous information—sample-driven data and analytic priors—allowing stable approximate inference without requiring access to the complete target density or joint samples. Experiments on Bayesian hierarchical models, joint density estimation, and large-scale astronomical data demonstrate its ability to accurately recover highly nonlinear structures, significantly outperforming existing methods and confirming its robustness and practical utility.

Normalized flows (NFs) remain underexploited for density estimation and generative modeling due to architectural complexity and limited scalability. This paper proposes TarFlow—a scalable NF architecture built upon a direction-alternating autoregressive Transformer that directly models pixel-level distributions within image patches. To enhance robustness and sample quality, we introduce Gaussian noise injection during training, post-training denoising, and a unified conditional/unconditional guidance mechanism. TarFlow is the first single-flow model to significantly surpass prior state-of-the-art methods on standard image likelihood estimation benchmarks, while simultaneously achieving sample fidelity and diversity on par with diffusion models. The implementation is publicly available.

This work addresses the limitations of flow-based generative models in modeling complex distributions—namely, inaccurate density estimation, unreliable likelihood evaluation, and lack of theoretical guarantees. Methodologically, we reformulate normalizing flows as Wasserstein gradient flows driven by neural ordinary differential equations (ODEs), unifying invertible transformations, density evolution, and sampling within a single geometric framework. We establish, for the first time, rigorous convergence guarantees for such flows, bridging optimal transport theory, ODE dynamics, and generative learning. By integrating differential-geometric modeling with Wasserstein metric analysis, our approach enables exact likelihood computation, efficient deterministic sampling, and interpretable generation mechanisms. Empirically, the framework significantly improves modeling stability and generalization across image and biosignal generation tasks, demonstrating both theoretical soundness and practical efficacy.

Existing expert prior elicitation methods struggle to model complex dependency structures and flexibly specify joint distributions. Method: We propose the first end-to-end, nonparametric joint prior learning framework based on normalizing flows. It transforms expert heuristic judgments into a differentiable density estimation task, employs deep normalizing flows to capture high-dimensional nonlinear dependencies, and integrates simulation-based inference for likelihood-free prior calibration. Contribution/Results: This work is the first to systematically introduce normalizing flows into expert elicitation, unifying support for both parametric and nonparametric, as well as independent and joint prior modeling; it further introduces a multi-stage diagnostic evaluation pipeline. Four simulation experiments demonstrate substantial improvements in prior density fidelity and expert interpretability, establishing a more powerful and transparent paradigm for Bayesian prior learning.

This work addresses the topological mismatch between standard normal latent variables and complex data distributions, which hinders the training efficiency and generative performance of normalizing flows. To mitigate this issue, the paper introduces, for the first time, a mixture of probabilistic principal component analyzers (MPPCA) as a learnable low-rank latent prior within the normalizing flow framework. This formulation effectively alleviates topological obstructions, simplifies the flow transformation architecture, and enables efficient initialization. The model is trained end-to-end by integrating the expectation-maximization (EM) algorithm with KL divergence minimization. Empirical evaluations on both tabular and image datasets demonstrate that the proposed approach significantly outperforms baseline methods, achieving faster convergence and superior sample quality.

This work addresses the computational challenge of estimating density ratios between intractable distributions by proposing a conditional-aware flow matching framework. By directly modeling the dynamics of density ratios along generative trajectories, the method circumvents the need for costly likelihood integrations over individual distributions. It introduces flow matching—a technique previously unexplored in this context—into density ratio estimation, leveraging a single ordinary differential equation (ODE)-based generative model to jointly characterize density ratios across multiple conditional distributions. This unified approach substantially enhances both computational efficiency and modeling flexibility. The framework achieves high-accuracy, closed-form density ratio estimates on synthetic data and demonstrates practical utility in single-cell genomics, where it is successfully applied to treatment effect estimation and batch correction.

This work addresses the exponential computational cost of modeling probability densities under continuously varying parameters by introducing Factorized Normalizing Flows (FNF). FNF represents parameter-dependent densities as a composition of a fixed, high-fidelity normalizing flow defined at a reference configuration and a factorizable polynomial transformation of the parameters. By leveraging an additive structure—with or without interaction terms—the approach enables independent learning of individual parameter effects and linearly combines multi-parameter responses, thereby circumventing the combinatorial explosion in the joint parameter space. FNF offers interpretability, linear scalability with respect to the number of parameters, and exact likelihood evaluation. In biaxial deformation experiments, it accurately reproduces true deformations and achieves state-of-the-art likelihood scores, demonstrating direct applicability to binning-free continuous density estimation tasks in fields such as high-energy physics.

Traditional normalizing flows struggle to capture the heavy-tailed nature of financial returns, leading to biased estimates of Value-at-Risk (VaR) and Expected Shortfall (ES). This work proposes Lévy-Flow, the first framework to integrate Lévy-driven heavy-tailed distributions—specifically Variance Gamma (VG) and Normal-Inverse Gaussian (NIG)—into normalizing flows. The model explicitly captures tail behavior while preserving exact likelihood computation and enabling efficient reparameterized sampling. Theoretically, it is shown that the proposed flow maintains the tail index under asymptotically linear transformations, which motivates the design of an Identity-tail Neural Spline Flow to faithfully preserve the base distribution’s tail shape. Empirical results on S&P 500 daily returns demonstrate that the VG flow reduces test negative log-likelihood by 69% compared to Gaussian flows and achieves well-calibrated 95% VaR, while the NIG flow yields the most accurate ES estimates.

Under limited sampling budgets, expectation estimation in flow matching models suffers from high variance due to rare, high-impact events under independent sampling. This work proposes an unbiased importance-weighted non-i.i.d. sampling framework—the first to integrate importance weighting into the flow matching generative process. Our method learns a residual velocity field guided by the score function, jointly reconstructing the target marginal distribution and estimating sample importance weights via diversity regularization. A score-based regularization term further enforces moderate separation of samples in high-density regions, mitigating off-manifold drift. Experiments demonstrate that the approach preserves estimator unbiasedness while significantly improving sample diversity and quality. Consequently, it yields more accurate and robust expectation estimates, enhancing both interpretability and reliability of flow matching model outputs.