design invertible transforms

Nonlinear dynamical systems often lack robust invertibility with respect to disturbances and initial state mismatches, limiting their reliability in control and generative modeling. This work proposes a robustly invertible nonlinear dynamical system framework that, for the first time, constructs a causal inverse system by composing strongly input–output monotone dynamic layers with static orthogonal layers. The resulting recurrent neural network ensures both forward and inverse dynamics are contracting and bi-Lipschitz, and yields a nonlinear minimum-phase/all-pass decomposition. A differentiable parameterization is achieved via bi-Lipschitz recurrent equilibrium networks (BiLipRENs), which significantly enhance performance in trajectory optimization, robust control, and complex distribution generation across tasks such as data-driven internal model control, dynamic surrogate loss learning, and signal-space normalizing flows.

This work addresses the challenge of robustly modeling invertibility for nonlinear dynamical systems. We introduce “bi-Lipschitz invertibility”—a novel definition requiring both forward and inverse mappings to be contractive (i.e., incrementally exponentially stable) and Lipschitz continuous. To realize this property, we propose BiLipREN, a neural architecture built upon recurrent equilibrium networks (RENs), incorporating orthogonal linear transformations and implicit layers, with explicit constraints enforcing bidirectional contraction. Its parameterization inherently supports minimum-phase/all-pass decomposition. We provide rigorous theoretical proof that BiLipREN strictly satisfies bi-Lipschitz invertibility. Numerical experiments demonstrate its strong robustness in reconstructing initial states and outputs under perturbations, significantly improving invertible stability and output distinguishability for nonlinear dynamical systems.

This work addresses the longstanding trade-off among expressiveness, smoothness, and computational efficiency in normalizing flow methods. We propose three families of analytically invertible bijections defined over the entire real line—rational cubic, sinh-based, and cubic polynomial—that simultaneously achieve global $C^\infty$ smoothness, closed-form inverses, strong expressivity, and tractable Jacobian determinants for the first time. Building upon these, we introduce a novel radial flow architecture that directly parameterizes radial coordinate transformations while preserving angular coordinates, yielding high numerical stability and geometric interpretability. Experiments demonstrate that our approach matches or exceeds spline flows on 1D/2D benchmarks and outperforms affine coupling flows in high-dimensional $\phi^4$ lattice field theory tasks with only one-thousandth the number of parameters, effectively mitigating mode collapse and enabling physics-informed, customizable modeling.

This work investigates how to reconstruct input images from neural network outputs to uncover the features underlying model decisions. To this end, two novel inversion methods are proposed: a forward inversion approach leveraging the input Jacobian matrix combined with root-finding algorithms, and a backward inversion technique that iteratively inverts layer-by-layer while injecting random vectors into the nullspace of each layer’s linear transformation. For the first time, high-fidelity input reconstructions are achieved on Transformers and linear sequence networks. The generated images, though appearing random, consistently yield near-100% classification confidence and densely span the feasible input space. This approach substantially outperforms existing methods and effectively exposes the model’s reliance on non-semantic features and its inherent vulnerabilities.

This paper addresses the challenge of integrating discrete logical structures—particularly integers—into continuous optimization and differentiable computation. Methodologically, it introduces a novel paradigm that embeds integers into smooth real-valued functions via an implicitly parameterized construction: for each integer (N), a function (f_N(t)) composed of alternating decaying Gaussian kernels is designed such that (N) is encoded as the smallest positive root of (int_0^t f_N(s),ds = 0)—i.e., its integral cancellation point. This yields a differentiable, implicit integer representation. The key contribution is the first formulation of integers as integral balance points of smooth functions, circumventing the non-differentiability of explicit discrete encodings. Theoretically, the encoding sequence (I(N) o 0) converges stably; numerically, (N) is recovered with high precision. The framework naturally generalizes to multidimensional integer tuples, providing a unified, differentiable interface for embedding discrete structures in neural networks, symbolic computation, and continuous optimization.

This work addresses the persistent challenge in process systems modeling of simultaneously achieving accuracy, simplicity, and physical interpretability—particularly in control applications where nonlinear expressiveness must be balanced against a preference for linear structures. The authors propose a convex hybrid modeling paradigm grounded in operator theory, which constrains models to interpretable subspaces or nonlinearly parameterized interpretable manifolds. By introducing a reparameterization technique based on “canonical features” in an augmented parameter space, the approach effectively integrates kernel methods with convex optimization. This framework enables the construction of kernel-based hybrid surrogate models over families of interpretable static and dynamic systems, significantly enhancing both predictive accuracy and computational efficiency while preserving physical interpretability across diverse process systems modeling scenarios.

This work addresses key challenges in post-hoc calibration—namely nonlinear miscalibration, poor scalability to large numbers of classes, and perturbation of original predictions—by proposing Invertible Logit Transformation (InvLT). InvLT applies a shared-parameter scalar MLP element-wise to pre-softmax logits and incorporates a paired inverse network with soft monotonicity constraints. This design achieves high expressiveness and strong class scalability without introducing class-dependent parameters or requiring model retraining, while rigorously preserving the original classification accuracy. Extensive experiments across diverse image classification benchmarks and model architectures demonstrate that InvLT consistently outperforms existing calibration methods on standard calibration metrics, all while maintaining the original predictive performance without degradation.

This work proposes a data-driven modeling approach for a broad class of nonlinear systems encompassing Volterra series, autoregressive models, and Hammerstein-type state-space realizations, without requiring explicit system identification. By extending Willems’ behavioral theory to vector-valued reproducing kernel Hilbert spaces (RKHS) and integrating minimal-norm interpolation with subspace identification techniques, the authors establish a unified framework for nonlinear system modeling. This study presents the first formulation of behavioral systems theory in vector-valued RKHS, thereby circumventing conventional identification procedures. The resulting framework enables direct application to simulation and control tasks and is applicable to a wide range of nonlinear dynamical systems.

This work proposes a general framework based on generalized singular value decomposition (GSVD) to represent modern neural networks as left-invertible, norm-preserving nonlinear mappings while preserving their input–output behavior. For the first time, GSVD theory is extended to generic neural network architectures, and a data-driven algorithm is introduced to estimate this representation from trained models, thereby aligning distances between the input space and the feature space. The resulting interpretable representation not only facilitates analyses of model invertibility and robustness but also demonstrates empirical effectiveness in adversarial perturbation detection. This approach establishes a theoretical foundation for diagnosing model bias and advancing invertible representation learning.

This study addresses the limited generalization of inverse dynamics models and inadequate uncertainty quantification in data-driven control of multirotor unmanned aerial vehicles. To this end, it introduces conditional invertible neural networks (CINNs) into this domain for the first time, leveraging incremental nonlinear dynamic inversion (INDI) as a teacher policy for supervised training. The proposed architecture employs rational quadratic spline coupling layers combined with invertible linear mixing to explicitly learn the probabilistic distribution of control inputs, thereby effectively capturing model uncertainty and revealing the critical influence of data coverage and command bandwidth on control failure. Experimental results on an X8 coaxial multirotor demonstrate an open-loop reproduction R² of 0.944 and a continuous ranked probability score (CRPS) of 0.0915; in closed-loop tests across 15 scenarios, the method achieves a position RMSE of 9.7 m and a tracking success rate of 47%, matching INDI’s performance while successfully identifying two dominant failure modes.