latent normalizing flow

This work addresses the topological mismatch between standard normal latent variables and complex data distributions, which hinders the training efficiency and generative performance of normalizing flows. To mitigate this issue, the paper introduces, for the first time, a mixture of probabilistic principal component analyzers (MPPCA) as a learnable low-rank latent prior within the normalizing flow framework. This formulation effectively alleviates topological obstructions, simplifies the flow transformation architecture, and enables efficient initialization. The model is trained end-to-end by integrating the expectation-maximization (EM) algorithm with KL divergence minimization. Empirical evaluations on both tabular and image datasets demonstrate that the proposed approach significantly outperforms baseline methods, achieving faster convergence and superior sample quality.

To address dimension redundancy, feature entanglement, and basis vector degeneracy in manifold learning, this paper proposes an invertible manifold modeling framework grounded in a *canonical intrinsic basis*. The method jointly learns a sparse, near-orthogonal, and non-degenerate latent-space eigenbasis—defined as the canonical intrinsic basis—by imposing ℓ₁-norm regularization on the manifold metric tensor to enable automatic basis selection. It integrates canonical normalization flows with manifold-constrained invertible mappings to preserve geometric consistency. Optimized within a maximum-likelihood framework, the approach significantly improves latent-space utilization and density estimation accuracy. Experiments demonstrate a 30–50% reduction in effective dimensionality and an average 12.6% improvement in Fréchet Inception Distance (FID) over state-of-the-art manifold-based normalizing flows, establishing superior performance across multiple benchmarks.

Normalized flows (NFs) remain underexploited for density estimation and generative modeling due to architectural complexity and limited scalability. This paper proposes TarFlow—a scalable NF architecture built upon a direction-alternating autoregressive Transformer that directly models pixel-level distributions within image patches. To enhance robustness and sample quality, we introduce Gaussian noise injection during training, post-training denoising, and a unified conditional/unconditional guidance mechanism. TarFlow is the first single-flow model to significantly surpass prior state-of-the-art methods on standard image likelihood estimation benchmarks, while simultaneously achieving sample fidelity and diversity on par with diffusion models. The implementation is publicly available.

Existing discrete data generation methods lack efficient and accurate modeling of categorical distributions. Method: This paper proposes the first flow-matching framework grounded in information geometry. It constructs a Riemannian structure on the categorical statistical manifold via the Fisher–Rao metric and defines optimal transport paths along geodesics, enabling exact likelihood computation without variational lower-bound constraints. Crucially, it integrates information geometry with flow matching for the first time, eliminating reliance on simplistic priors (e.g., uniform or independent distributions). Contribution/Results: We develop a natural-gradient-driven geodesic flow-matching algorithm that supports diffeomorphic mappings and invertible modeling. Empirically, our method substantially outperforms state-of-the-art discrete diffusion and flow models on image, text, and biological sequence generation tasks—simultaneously improving both sample quality and log-likelihood accuracy.

Existing expert prior elicitation methods struggle to model complex dependency structures and flexibly specify joint distributions. Method: We propose the first end-to-end, nonparametric joint prior learning framework based on normalizing flows. It transforms expert heuristic judgments into a differentiable density estimation task, employs deep normalizing flows to capture high-dimensional nonlinear dependencies, and integrates simulation-based inference for likelihood-free prior calibration. Contribution/Results: This work is the first to systematically introduce normalizing flows into expert elicitation, unifying support for both parametric and nonparametric, as well as independent and joint prior modeling; it further introduces a multi-stage diagnostic evaluation pipeline. Four simulation experiments demonstrate substantial improvements in prior density fidelity and expert interpretability, establishing a more powerful and transparent paradigm for Bayesian prior learning.

This paper addresses the problem of efficient and theoretically sound density estimation on injective manifolds. We propose Random Projection Flows (RPFs), a class of invertible generative models that project high-dimensional data onto lower-dimensional manifolds via random semi-orthogonal matrices drawn from the Haar distribution, enabling exact, differentiable, and reversible dimensionality reduction. Leveraging Riemannian geometry, we derive a closed-form volume correction term for the change of variables, yielding a tractable, training-free, plug-and-play probabilistic model. RPFs constitute the first unification of random projection theory with normalizing flows, preserving invertibility and theoretical rigor while drastically reducing computational overhead. Experiments demonstrate competitive performance in generative modeling tasks across diverse benchmarks. By bridging theoretical foundations with practical efficiency, RPFs establish a strong, principled baseline for unsupervised learning.

Existing density modeling approaches suffer from high training costs, slow inference, approximate likelihood evaluation, mode collapse, or architectural constraints (e.g., enforced bijectivity). This paper proposes a flexible density estimation framework based on learned latent variable marginalization: by introducing a trainable latent distribution and performing Monte Carlo integration for marginalization, the method supports arbitrary neural architectures, exact likelihood computation, and efficient forward/backward sampling. It is the first to integrate latent variable modeling with variational marginalization—thereby circumventing manifold assumptions and invertibility requirements—enabling effective modeling of multimodal distributions and low-dimensional manifolds. Experiments on synthetic data, image latent spaces, positive-definite matrix distributions, and simulation-based inference tasks demonstrate speedups of several orders of magnitude in both training and inference over state-of-the-art methods, with no mode collapse.

This work addresses the challenge in unsupervised representation learning of simultaneously achieving semantic interpretability and cross-run stability. The authors propose Entropy-Ordered Flows (EOFlows), a novel framework that, for the first time, integrates explanatory entropy into normalizing flows. By sorting latent dimensions according to their explanatory entropy after training, EOFlows adaptively disentangles core semantic factors from fine-grained noise, enabling variable-rate compression and disentanglement without pre-specifying the latent dimensionality. The method combines likelihood-based training, local Jacobian regularization, and noise augmentation, synergistically integrating independent mechanism analysis, principal component flows, and manifold entropy measures. Evaluated on CelebA, EOFlows successfully extracts highly interpretable semantic features, significantly improving both compression fidelity and denoising performance.

This work addresses the challenges of modeling complex dependencies and mitigating redundancy in high-dimensional parameter spaces for discrete data generation. It introduces, for the first time, a Riemannian geometric structure with isometric properties into the exponential parameter space of product manifolds over categorical distributions, thereby constructing a low-dimensional latent subspace. By leveraging the Riemannian metric, geodesics within this subspace become straight lines, enabling consistent and efficient flow-matching training. The proposed approach substantially reduces the dimensionality of latent variables while preserving strong representational capacity for discrete data distributions. Experimental results demonstrate that the model achieves accurate and efficient discrete data generation using a significantly lower-dimensional latent space, effectively balancing computational efficiency with modeling performance.

Modeling multimodal posterior distributions in high-dimensional inverse problems remains challenging—particularly due to spurious probability bridges induced by unimodal base distributions. Method: We propose a likelihood-weighted importance sampling normalizing flow (LWIS-NF) that requires no posterior samples for training. Our approach explicitly addresses the critical role of base distribution topology in posterior modeling and introduces a mixture-of-Gaussians initialization strategy, where the number of components matches the expected number of posterior modes, thereby guiding the flow to learn disconnected, multimodal supports. Contribution/Results: By integrating the expressive power of normalizing flows, bias correction via likelihood-weighted importance sampling, and structure-aware initialization, LWIS-NF achieves significantly improved posterior reconstruction fidelity on 2D and 3D multimodal benchmark tasks. Quantitatively, it reduces Wasserstein distance and KL divergence by 30%–50% compared to state-of-the-art alternatives.