normalizing flows

Existing expert prior elicitation methods struggle to model complex dependency structures and flexibly specify joint distributions. Method: We propose the first end-to-end, nonparametric joint prior learning framework based on normalizing flows. It transforms expert heuristic judgments into a differentiable density estimation task, employs deep normalizing flows to capture high-dimensional nonlinear dependencies, and integrates simulation-based inference for likelihood-free prior calibration. Contribution/Results: This work is the first to systematically introduce normalizing flows into expert elicitation, unifying support for both parametric and nonparametric, as well as independent and joint prior modeling; it further introduces a multi-stage diagnostic evaluation pipeline. Four simulation experiments demonstrate substantial improvements in prior density fidelity and expert interpretability, establishing a more powerful and transparent paradigm for Bayesian prior learning.

This work addresses the topological mismatch between standard normal latent variables and complex data distributions, which hinders the training efficiency and generative performance of normalizing flows. To mitigate this issue, the paper introduces, for the first time, a mixture of probabilistic principal component analyzers (MPPCA) as a learnable low-rank latent prior within the normalizing flow framework. This formulation effectively alleviates topological obstructions, simplifies the flow transformation architecture, and enables efficient initialization. The model is trained end-to-end by integrating the expectation-maximization (EM) algorithm with KL divergence minimization. Empirical evaluations on both tabular and image datasets demonstrate that the proposed approach significantly outperforms baseline methods, achieving faster convergence and superior sample quality.

Normalized flows (NFs) remain underexploited for density estimation and generative modeling due to architectural complexity and limited scalability. This paper proposes TarFlow—a scalable NF architecture built upon a direction-alternating autoregressive Transformer that directly models pixel-level distributions within image patches. To enhance robustness and sample quality, we introduce Gaussian noise injection during training, post-training denoising, and a unified conditional/unconditional guidance mechanism. TarFlow is the first single-flow model to significantly surpass prior state-of-the-art methods on standard image likelihood estimation benchmarks, while simultaneously achieving sample fidelity and diversity on par with diffusion models. The implementation is publicly available.

Normalized flow generative models suffer from interpolation paths deviating from the data manifold, primarily due to norm drift induced by Gaussian base distributions in latent space. To address this, we propose a norm-constrained base distribution reconstruction framework—introducing Dirichlet and von Mises–Fisher distributions into normalized flows for the first time. These distributions explicitly constrain latent variables to the unit simplex or unit hypersphere, respectively, ensuring geometrically consistent interpolation trajectories. Our method requires no architectural modifications to the flow network and provides an interpretable, unambiguous interpolation criterion, effectively overcoming interpolation distortion inherent to the Gaussian assumption. Experiments demonstrate consistent improvements over baselines across all major evaluation metrics: bits/dim, Fréchet Inception Distance (FID), and Kernel Inception Distance (KID). Interpolation quality is significantly enhanced while strictly preserving original generation performance.

To address performance degradation in non-stationary time series forecasting caused by distributional shift, this paper proposes a decoupled modeling framework that separates distribution correction from forecasting and introduces a bilevel optimization paradigm for joint learning. Its core innovation is Instance Normalization Flow (IN-Flow)—a reversible, bidirectional, and highly expressive temporal distribution transformation network explicitly designed for forecasting, overcoming the limitation of conventional normalizing flows restricted to generative tasks. IN-Flow integrates instance normalization layers with stacked invertible neural networks: the outer level optimizes distribution transformation, while the inner level optimizes forecasting, ensuring compatibility with arbitrary forecasting architectures and eliminating reliance on statistical assumptions. Extensive experiments on synthetic and diverse real-world datasets demonstrate significant improvements over state-of-the-art methods, strong robustness to unseen distribution shifts, and simultaneous gains in both predictive accuracy and generalization capability.

Traditional normalizing flows (NFs) are constrained by the requirement that forward transformations must be explicitly invertible, necessitating inefficient autoregressive decoding for inverse sampling—a key performance bottleneck. This paper proposes Bidirectional Normalizing Flows (BiFlow), the first NF framework to relax explicit invertibility: it jointly learns a forward transformation and a trainable approximate inverse mapping, enabling non-causal, highly parallel reverse modeling. BiFlow integrates Transformer architectures with autoregressive flow components, supporting flexible loss design and non-causal network topologies. On ImageNet, BiFlow achieves superior generation quality compared to prior NF methods, accelerates sampling by two orders of magnitude—matching the efficiency of 1-NFE approaches—and establishes new state-of-the-art performance among NF-based models.

This work addresses the exponential computational cost of modeling probability densities under continuously varying parameters by introducing Factorized Normalizing Flows (FNF). FNF represents parameter-dependent densities as a composition of a fixed, high-fidelity normalizing flow defined at a reference configuration and a factorizable polynomial transformation of the parameters. By leveraging an additive structure—with or without interaction terms—the approach enables independent learning of individual parameter effects and linearly combines multi-parameter responses, thereby circumventing the combinatorial explosion in the joint parameter space. FNF offers interpretability, linear scalability with respect to the number of parameters, and exact likelihood evaluation. In biaxial deformation experiments, it accurately reproduces true deformations and achieves state-of-the-art likelihood scores, demonstrating direct applicability to binning-free continuous density estimation tasks in fields such as high-energy physics.

This work proposes a novel approach that integrates normalizing flows with stratified sampling to estimate expectations without relying on restrictive (semi-)parametric distributional assumptions, such as Gaussian or Gaussian mixture models, which can introduce substantial bias when misspecified. By leveraging the expressive power of neural networks, the method flexibly captures complex, unknown data distributions, thereby overcoming the limitations of traditional parametric frameworks. Empirical evaluations demonstrate that the proposed estimator significantly reduces Monte Carlo uncertainty in high-dimensional settings—specifically in 30- and 128-dimensional problems—and achieves marked improvements in both accuracy and stability compared to conventional Monte Carlo estimators and Gaussian mixture model-based approaches.

This work addresses the challenging problem of density estimation for complex target distributions that combine an analytically tractable component with a non-analytic component accessible only through samples from a simulator or dataset, where direct sampling or closed-form evaluation is infeasible. The authors propose a two-stage normalizing flow framework: the first stage learns the density of the non-analytic component from samples, and the second stage integrates this estimate with the analytic term to reconstruct the full target distribution, enabling efficient sampling and density evaluation. This approach uniquely unifies heterogeneous information—sample-driven data and analytic priors—allowing stable approximate inference without requiring access to the complete target density or joint samples. Experiments on Bayesian hierarchical models, joint density estimation, and large-scale astronomical data demonstrate its ability to accurately recover highly nonlinear structures, significantly outperforming existing methods and confirming its robustness and practical utility.

This study addresses the challenge of preserving the full conditional distribution of predictors given a response variable in dimension reduction. To this end, it proposes a likelihood-based sufficient dimension reduction (SDR) framework that introduces conditional normalizing flows to the SDR literature for the first time. The method jointly learns a linear projection and a flexible conditional density by maximizing the conditional log-likelihood, employing monotonic rational quadratic spline flows to model complex conditional distributions. The approach is grounded in an interpretable mutual information objective and complemented by a neural Gaussian SDR variant as an auxiliary model. Theoretical analysis establishes Fisher consistency, and empirical evaluations across diverse simulation settings and the UTKFace age prediction task demonstrate accurate recovery of the central subspace, significantly outperforming existing SDR methods and neural Gaussian baselines.