continuous-time generative modeling

This paper investigates the theoretical behavior of generative models under finite training samples. For both deterministic and stochastic generation processes, it derives closed-form solutions for the velocity field and score function within a stochastic interpolation framework—revealing that the former exactly recovers training samples, while the latter corresponds to adding Gaussian noise to them. It introduces the first formal definitions of underfitting and overfitting for generative models, proving that, in the presence of model estimation error, stochastic generation amounts to convex combinations of training samples corrupted by a mixture of noise sources. These theoretical findings are empirically validated on downstream classification tasks, confirming that the characterized noise structure aligns with observed generalization performance. The core contribution is an analytical theory of generative processes under finite-sample regimes, unifying the explanatory frameworks for sample recovery and perturbation, and providing verifiable criteria for diagnosing underfitting and overfitting.

This work demonstrates that diffusion models, score-based generative models, and flow matching methods—despite their apparent formal differences—share a unified continuous-time generative mechanism. By constructing a measure-theoretic framework, the paper unifies these approaches as learning time-dependent vector fields that transport a reference distribution to the data distribution, with distributional evolution governed by the continuity equation and the Fokker–Planck equation. It establishes, for the first time under a common perspective, the equivalence and distinctions among the three paradigms, clarifies the relationship between probability flow ODEs and stochastic backward dynamics, and identifies flow matching as essentially a velocity field regression problem. The study further provides a systematic comparison of objective functions, sampling strategies, and discretization errors, links the framework to Schrödinger bridges and entropy-regularized optimal transport, and summarizes theoretical guarantees and open challenges regarding approximation capacity, stability, and scalability.

This work addresses the challenge of developing a modality-agnostic, unified generative modeling framework that generalizes and unifies Markovian generative approaches. We propose Generator Matching—a principled framework grounded in arbitrary Markov processes (including continuous diffusion, flow, discrete transition, and jump processes)—which models data distributions by rigorously aligning conditional and marginal generators. Our contributions are threefold: (i) the first unified treatment of diffusion models, flow matching, and discrete diffusion under a single theoretical umbrella; (ii) the first systematic extension of generative modeling to non-standard jump processes; and (iii) support for rigorous superposition of Markov generators and joint multimodal modeling. Experiments demonstrate substantial performance gains on image and multimodal generation tasks, with superposed jump processes delivering significant empirical improvements.

Discrete-state-space generative models—e.g., for small molecules, DNA, and protein sequences—lack principled, controllable guidance mechanisms. Existing continuous-domain guidance paradigms do not generalize to discrete spaces, hindering attribute-controlled generation. Method: This paper introduces the first general, differentiable guidance framework for discrete generative models based on continuous-time Markov processes, unifying discrete diffusion and flow-matching architectures. It overcomes the fundamental incompatibility of continuous guidance with discrete state spaces by establishing a theoretically grounded guidance theory for discrete domains. The framework enables arbitrary differentiable guidance objectives without model retraining, leveraging probability path reweighting and gradient-driven discrete sampling. Contribution/Results: Experiments demonstrate substantial improvements in target property satisfaction rates and sample diversity across diverse biomolecular generation tasks, while maintaining flexibility and strong generalization across guidance objectives and model architectures.

This work investigates the theoretical underpinnings of memorization and overfitting in stochastic interpolation generative models. Focusing on continuous-time stochastic differential equations and their Euler discretization, it provides the first rigorous theoretical definitions of overfitting and underfitting in generative modeling and derives closed-form expressions for the optimal velocity field and score function. The analysis reveals that generated samples can be expressed as training samples perturbed by three controllable error terms, whose bias is jointly determined by the discretization step size and estimation error. Synthetic experiments corroborate the theoretical prediction that generated samples cluster around the training data distribution, highlighting the critical roles of error accumulation and noise modeling in the model’s reconstruction capability.

This work addresses the theoretical gap in sample complexity analysis for flow-matching generative models. Unlike prior studies relying on empirical risk minimization (ERM) assumptions, we establish the first end-to-end upper bound on sample complexity without such assumptions. Methodologically, we model the continuous flow via ordinary differential equations and parameterize the velocity field using neural networks; we then introduce a triple-error decomposition framework—comprising neural approximation error, statistical error, and optimization error—and rigorously analyze its convergence. Our theoretical analysis shows that $O(varepsilon^{-4})$ samples suffice to achieve $O(varepsilon)$ generative accuracy in the Wasserstein-2 distance. This constitutes the first rigorous, non-ERM-dependent sample complexity guarantee for flow matching, filling a critical theoretical void. Moreover, our result provides foundational insights for efficient training and generalization analysis of flow-based generative models.

Existing diffusion model sampling methods typically employ a globally uniform time-step schedule, disregarding the varying generation complexity across individual samples and thereby limiting the quality of few-step sampling. This work proposes an instance-aware discretization framework that, for the first time, introduces sample-adaptive mechanisms into time-step scheduling. By leveraging input-dependent priors to dynamically adjust time-step allocation and extending gradient-based discretization search to conditional generation settings, the method achieves significant improvements in few-step sampling quality across diverse tasks—including synthetic data, pixel-space images, latent-space images, and video generation. Built upon probability flow ODEs and numerical solvers, the approach incurs minimal tuning cost and negligible inference overhead.

This work investigates whether standard continuous-time generative models can effectively capture the probability distribution of highly sparse, globally constrained discrete structures—such as Sudoku puzzles. Treating complete Sudoku grids as a discrete subset within a continuously relaxed space, we train flow-matching and score-based generative models, and systematically compare the performance of ODE-, SDE-, and DDPM-style sampling in both unconditional generation and constraint-satisfaction tasks. We demonstrate for the first time that such models can assign non-zero probability mass to globally constrained combinatorial structures and satisfy constraints through stochastic sampling. Empirical results show that score-based stochastic sampling is the most stable, while DDPM-style sampling achieves the highest efficacy. Although less sample-efficient than specialized solvers, our approach establishes the feasibility of using general-purpose probabilistic generative models as Sudoku solvers, thereby extending the applicability of continuous generative modeling to discrete reasoning problems.

This work addresses the lack of a general framework for modeling time-varying latent states in existing generative models, which often rely on auxiliary stochastic processes that are difficult to sample. The authors propose a novel approach that treats observation generation as a deterministic mapping of a tractable Markov process, employing an image-space stochastic process generator whose one-time marginal distribution matches that of a target projected process. The key innovation lies in extending Generator Matching—previously limited to static latent variables—to time-varying latent processes for the first time. By integrating stochastic process theory, Markov projections, and flow matching techniques, the method establishes a unified generative modeling framework. This framework not only subsumes existing models with discrete latent processes as special cases but also accommodates a broader class of time-varying latent conditions while rigorously ensuring consistency between the generated and target marginal distributions.

This work proposes a novel generative paradigm that circumvents the limitations of conventional models, which rely on mapping from a noise prior to data and often fail to preserve the intrinsic characteristics of the true data distribution. Instead, the approach leverages discrete stochastic dynamics that leave the data distribution invariant, initializing directly from the data support set. By integrating probability-conserving sampling with Metropolis-adjusted Langevin dynamics and predictor-corrector flows, the method operates directly on existing pretrained flow model checkpoints without requiring retraining, thereby enhancing sampling quality universally. Empirical evaluations on Swiss-roll, ImageNet-256, and Oxford Flowers-102 demonstrate consistent and significant improvements in generation fidelity over standard sampling techniques.