train normalizing flows

This work addresses the exponential computational cost of modeling probability densities under continuously varying parameters by introducing Factorized Normalizing Flows (FNF). FNF represents parameter-dependent densities as a composition of a fixed, high-fidelity normalizing flow defined at a reference configuration and a factorizable polynomial transformation of the parameters. By leveraging an additive structure—with or without interaction terms—the approach enables independent learning of individual parameter effects and linearly combines multi-parameter responses, thereby circumventing the combinatorial explosion in the joint parameter space. FNF offers interpretability, linear scalability with respect to the number of parameters, and exact likelihood evaluation. In biaxial deformation experiments, it accurately reproduces true deformations and achieves state-of-the-art likelihood scores, demonstrating direct applicability to binning-free continuous density estimation tasks in fields such as high-energy physics.

This work addresses the topological mismatch between standard normal latent variables and complex data distributions, which hinders the training efficiency and generative performance of normalizing flows. To mitigate this issue, the paper introduces, for the first time, a mixture of probabilistic principal component analyzers (MPPCA) as a learnable low-rank latent prior within the normalizing flow framework. This formulation effectively alleviates topological obstructions, simplifies the flow transformation architecture, and enables efficient initialization. The model is trained end-to-end by integrating the expectation-maximization (EM) algorithm with KL divergence minimization. Empirical evaluations on both tabular and image datasets demonstrate that the proposed approach significantly outperforms baseline methods, achieving faster convergence and superior sample quality.

Existing expert prior elicitation methods struggle to model complex dependency structures and flexibly specify joint distributions. Method: We propose the first end-to-end, nonparametric joint prior learning framework based on normalizing flows. It transforms expert heuristic judgments into a differentiable density estimation task, employs deep normalizing flows to capture high-dimensional nonlinear dependencies, and integrates simulation-based inference for likelihood-free prior calibration. Contribution/Results: This work is the first to systematically introduce normalizing flows into expert elicitation, unifying support for both parametric and nonparametric, as well as independent and joint prior modeling; it further introduces a multi-stage diagnostic evaluation pipeline. Four simulation experiments demonstrate substantial improvements in prior density fidelity and expert interpretability, establishing a more powerful and transparent paradigm for Bayesian prior learning.

Normalized flow generative models suffer from interpolation paths deviating from the data manifold, primarily due to norm drift induced by Gaussian base distributions in latent space. To address this, we propose a norm-constrained base distribution reconstruction framework—introducing Dirichlet and von Mises–Fisher distributions into normalized flows for the first time. These distributions explicitly constrain latent variables to the unit simplex or unit hypersphere, respectively, ensuring geometrically consistent interpolation trajectories. Our method requires no architectural modifications to the flow network and provides an interpretable, unambiguous interpolation criterion, effectively overcoming interpolation distortion inherent to the Gaussian assumption. Experiments demonstrate consistent improvements over baselines across all major evaluation metrics: bits/dim, Fréchet Inception Distance (FID), and Kernel Inception Distance (KID). Interpolation quality is significantly enhanced while strictly preserving original generation performance.

This study addresses the limited expressiveness and theory-practice gap of normalizing flows arising from architectural constraints. Methodologically, it integrates distributional theory analysis, decoupled flow architecture modeling, and function approximation arguments under condition-number constraints. Theoretically, it establishes, for the first time under well-conditioned neural network assumptions, the universal distributional approximation capability of coupling-based flows (e.g., RealNVP); concurrently, it proves the intrinsic non-universality of volume-preserving flows—demonstrating they can only model specific distribution subclasses—and proposes a repair mechanism based on invertible perturbations and conditional volume scaling. These results bridge the gap between theoretical expectations and empirical performance, providing a new paradigm for designing normalizing flow models that simultaneously guarantee theoretical soundness and practical efficacy.

This work systematically investigates the impact of loss weighting strategies and output parameterizations on model performance in flow matching. Through numerical experiments on both synthetic data with controllable geometric structures and real-world images, the study disentangles their interaction effects across varying data manifold dimensions, model architectures, and dataset scales, using PSNR and FID as evaluation metrics. The analysis reveals, for the first time, how the optimal choice of loss weighting and parameterization depends critically on the intrinsic structure of the data. Building on these insights, the authors formulate practical design principles that substantially improve denoising accuracy and generation quality.

This work proposes Normalized Flow Matching (NFM), a novel generative training framework that overcomes the limitations of traditional flow matching methods, which are constrained by independent or optimal transport couplings and struggle to capture complex dependencies between noise and data. NFM leverages the bijective structure of a pre-trained autoregressive normalizing flow (AR-NF) to distill a high-quality deterministic coupling, which is then used to train a lightweight student flow model. By integrating autoregressive normalizing flows, flow matching, and knowledge distillation, the method enables efficient and effective learning. Experimental results demonstrate that the student model significantly outperforms existing flow matching approaches in generation quality and even surpasses the teacher AR-NF model.

This work addresses a key limitation in existing flow matching theory, which assumes that the target distribution possesses a full-dimensional smooth density—an assumption violated by data supported on low-dimensional manifolds, despite empirical success in such settings. Focusing on flow matching with linear interpolation, this paper establishes the first non-asymptotic convergence guarantees when the target distribution is supported on a smooth manifold. By modeling the dynamics via ordinary differential equations, employing nonparametric estimation on the manifold, and conducting a careful error propagation analysis, the authors demonstrate that the convergence rate depends only on the intrinsic dimension of the manifold and the smoothness of the target distribution restricted to it. This result circumvents the curse of dimensionality and achieves a near-minimax optimal rate.

This work proposes a novel approach that integrates normalizing flows with stratified sampling to estimate expectations without relying on restrictive (semi-)parametric distributional assumptions, such as Gaussian or Gaussian mixture models, which can introduce substantial bias when misspecified. By leveraging the expressive power of neural networks, the method flexibly captures complex, unknown data distributions, thereby overcoming the limitations of traditional parametric frameworks. Empirical evaluations demonstrate that the proposed estimator significantly reduces Monte Carlo uncertainty in high-dimensional settings—specifically in 30- and 128-dimensional problems—and achieves marked improvements in both accuracy and stability compared to conventional Monte Carlo estimators and Gaussian mixture model-based approaches.

This work addresses the challenge of accurately modeling complex state evolution in high-order dynamical systems under irregularly sampled observations—a setting where existing flow matching methods falter due to their reliance on linear interpolation. To overcome this limitation, we propose SplineFlow, the first framework to integrate B-spline basis functions into flow matching. By leveraging B-spline interpolation, SplineFlow constructs smooth and stable conditional trajectories that satisfy multi-marginal constraints, thereby capturing intricate temporal dynamics more faithfully. The method combines continuous normalizing flows with multi-marginal constrained optimization, achieving significant performance gains over current baselines across diverse tasks, including deterministic and stochastic dynamical systems as well as single-cell trajectory inference, particularly in scenarios involving high-order dynamics and irregular sampling.