regularization scheme design

This paper identifies a critical implicit bias arising from numerical integration scheme selection in learning dynamical systems from sparse temporal observations: even with perfect data fit, inappropriate schemes (e.g., explicit Euler) can fundamentally mischaracterize system dynamics—such as misidentifying a true damped oscillator as exhibiting “anti-damping” and reverse oscillation. Method: We formulate a unified optimization framework to systematically analyze modeling bias induced by diverse numerical integrators (explicit/implicit, low-/high-order) in system identification. Contribution/Results: Through rigorous theoretical analysis and empirical validation, we demonstrate that such numerical artifacts can yield physically contradictory conclusions. Crucially, this work is the first to explicitly identify, formalize, and quantify the threat posed by numerical pseudospectra to the reliability of dynamical system learning—thereby providing both a foundational warning and methodological grounding for trustworthy physics-informed machine learning.

Ill-posed inverse problems suffer from instability and non-uniqueness, posing fundamental challenges for reliable reconstruction and generalization. Method: This work establishes a cross-disciplinary unifying framework for regularization, systematically connecting conceptual developments across inverse problems, statistics, machine learning, and deep learning. Adopting a question-driven exposition, it analyzes both explicit regularization (e.g., Tikhonov, Lasso) and implicit mechanisms (e.g., gradient descent trajectories, parameter initialization, architectural inductive biases). Contribution/Results: We propose a novel regularization concept mapping system, formally integrating implicit regularization in deep learning into the classical inverse problem theory spectrum for the first time. We uncover its fundamental role in governing model interpretability, robustness, and generalization—revealing mechanistic links between optimization dynamics, architecture design, and solution stability. The framework provides a principled foundation for theoretical modeling, algorithmic development, and pedagogical practice in ill-posed problems.

In training nonlinear state-space models, state trajectories often undergo severe distortion, leading to sparse state-space coverage and structural deformation—thereby compromising interpretability and robustness. To address this, we propose a data-distribution-aware trajectory regularization method tailored for locally affine state-space models. Our approach introduces two distribution-aware regularizers: (i) an affine consistency constraint imposed on local model parameters to preserve local linearity, and (ii) a density-aware uniform coverage penalty applied along state trajectories to encourage balanced exploration of the state space. Integrating system modeling priors with experimental design principles, our method seamlessly integrates into spatially guided training frameworks. Experiments on standard system identification benchmarks demonstrate that our method significantly improves the quality of state-space distribution, enhances training stability, boosts model interpretability, and strengthens robustness against input perturbations and distributional shifts.

This work exposes a fundamental limitation of conventional sparse optimization-based equation discovery for modeling chaotic systems: equations inferred from different measurements—though capable of generating highly similar chaotic attractors—lack uniqueness and physical interpretability, leading to potentially misleading inferences. Integrating sparse regression, Koopman spectral analysis, and numerical simulations, the study systematically examines multiple chaotic systems and reveals that small-magnitude Koopman eigenvalues are highly sensitive to measurement perturbations, whereas only large-magnitude eigenvalues remain robust. This undermines the prevailing “unique correct equation” paradigm. The key contribution is the first operator-spectral proof that the deterministic assumption underlying equation discovery fails for chaotic dynamics. Consequently, the paper argues that for strongly nonlinear, highly sensitive systems, end-to-end data-driven modeling—particularly machine learning approaches—should supersede the pursuit of explicit differential equations.

To address the prevalent reward collapse problem in diffusion model fine-tuning, this paper proposes an entropy-regularized stochastic control framework and— for the first time—rigorously extends it to general *f*-divergence regularization. Methodologically, we formulate a continuous-time stochastic control model, integrating Itô calculus with variational inference to derive a computationally tractable and provably convergent optimal control policy. Theoretically, we establish that the proposed regularization effectively mitigates reward collapse; empirically, it significantly improves both sample quality and diversity. Key contributions include: (1) the first rigorous stochastic control analysis framework specifically designed for diffusion model fine-tuning; (2) a unified generalization of entropy regularization to arbitrary *f*-divergences, substantially enhancing methodological generality and robustness; and (3) a practical fine-tuning paradigm implementable under multiple divergence metrics.

This work addresses the persistent challenge in process systems modeling of simultaneously achieving accuracy, simplicity, and physical interpretability—particularly in control applications where nonlinear expressiveness must be balanced against a preference for linear structures. The authors propose a convex hybrid modeling paradigm grounded in operator theory, which constrains models to interpretable subspaces or nonlinearly parameterized interpretable manifolds. By introducing a reparameterization technique based on “canonical features” in an augmented parameter space, the approach effectively integrates kernel methods with convex optimization. This framework enables the construction of kernel-based hybrid surrogate models over families of interpretable static and dynamic systems, significantly enhancing both predictive accuracy and computational efficiency while preserving physical interpretability across diverse process systems modeling scenarios.

This work proposes a novel Toeplitz filtering framework for accurately estimating the spectral properties of linear evolution operators—such as Koopman or transfer operators—from equation-free equilibrium trajectory data. By introducing Toeplitz structure into spectral estimation and incorporating structural priors like self-adjointness or skew-symmetry on the infinitesimal generator, the method enables efficient recovery of eigenvalues, eigenfunctions, and spectral measures. Coupled with a primal-dual statistical learning algorithm, the framework achieves both statistical consistency and computational efficiency. Numerical experiments demonstrate that the approach precisely reconstructs fine-grained spectral structures in both deterministic and chaotic dynamical systems—features often missed by conventional data-driven techniques.

This study addresses the challenge of global linear modeling and control for highly nonlinear dynamical systems by leveraging Koopman operator theory. By introducing observable functions, the nonlinear dynamics are lifted into a higher-dimensional space where they admit an approximately linear representation. A data-driven surrogate model is constructed through a synergistic integration of Extended Dynamic Mode Decomposition (EDMD), kernelized EDMD, and machine learning techniques. The work innovatively extends the Koopman framework to input-affine systems, proposing a unified modeling approach and a corresponding Koopman-based Model Predictive Control (MPC) design methodology. Numerical simulations demonstrate that the proposed method achieves high-fidelity modeling accuracy and effective closed-loop control performance. Full reproducibility is supported by the accompanying open-source implementation.

This study addresses the numerical ill-conditioning commonly encountered in dictionary learning for dynamical equation discovery in systems biology, where highly correlated candidate functions degrade model identification accuracy. The work systematically investigates the impact of multicollinearity in sparse regression on biological dynamical modeling, revealing that even a small subset of terms can induce severe ill-conditioning. Through comparative analysis of orthogonal polynomial bases and monomial bases under varying data distributions—supported by condition number assessments and numerical experiments on benchmark systems biology models—the study demonstrates that orthogonal bases substantially improve conditioning, numerical stability, and model recovery accuracy only when their associated weight functions align with the data sampling distribution.

This work proposes a stabilization framework based on score-based generative models to address non-physical instabilities and structural distortions commonly encountered in the numerical solution of time-dependent partial differential equations (PDEs). For the first time, score-based generative models are introduced into PDE numerical stabilization, where a conditional stabilizing operator with manifold-contracting properties is constructed by learning the physically admissible solution manifold. This operator corrects intermediate solutions during time integration. Numerical experiments demonstrate that the method significantly enhances robustness for convection, Korteweg–de Vries (KdV), nonlinear Schrödinger, and Burgers equations, effectively suppressing spurious oscillations while preserving essential dynamical features.