jacobian computation

This work addresses the optimal control problem for high-dimensional, nonlinear, and analytically intractable dynamical systems—including discrete/continuous-time and deterministic/stochastic settings. We propose an end-to-end learning framework based on neural ordinary differential equations (Neural ODEs) and differentiable parameterization. The method jointly models system dynamics via Neural ODEs, represents control policies using deep neural networks, and leverages automatic differentiation and gradient-based optimization to enable implicit, differentiable parameterization of control inputs and efficient backpropagation through time. Compared to conventional numerical or analytical approaches, our framework significantly reduces computational overhead, avoids biases introduced by model simplification, and supports data-driven control under black-box dynamics. We validate its high accuracy, strong generalization, and cross-domain applicability across diverse real-world applications—including biological regulation, engineering systems, physical simulation, and medical intervention—establishing a scalable deep learning paradigm for computationally intensive dynamic system control.

This work proposes a randomized, unbiased vector-Jacobian product (VJP) approximation method with minimal variance to replace exact computations in backpropagation, aiming to reduce the computational and memory costs of training deep neural networks. The approach achieves theoretically optimal estimation under sparsity constraints and establishes a principled trade-off between approximation accuracy and per-iteration training cost. Empirical evaluations on multilayer perceptrons, BagNets, and Vision Transformers demonstrate that the method substantially lowers training overhead while preserving model accuracy almost entirely.

Physics-informed neural networks (PINNs) often suffer from slow convergence and poor stability when solving partial differential equations on irregular domains, primarily due to normalization mismatch, ineffective boundary enforcement, and imbalanced loss terms. Existing coordinate-mapping approaches rely on manually generated meshes and simple geometries, limiting generalizability and integration into end-to-end learning frameworks. This paper proposes JacobiNet: an end-to-end differentiable coordinate transformation framework that employs a lightweight MLP to learn a smooth, invertible mapping from complex physical domains to a unit reference domain. Crucially, it leverages automatic differentiation to implicitly compute the Jacobian determinant—eliminating the need for mesh generation or explicit derivative construction. JacobiNet further supports hard boundary constraints and adaptive loss balancing. Experiments demonstrate significant improvements: relative L² errors reduce to 0.013–0.039 (average 18.3× reduction), vascular-domain accuracy increases by 3.65×, and computational time decreases by over 10× compared to baseline PINNs.

Understanding the convergence behavior of gradient flow in deep neural network training remains challenging due to the non-convexity and high dimensionality of parameter space. Method: Under the full-rank condition of the Jacobian matrix, we establish a rigorous equivalence—via time reparameterization—between the (constrained) Euclidean gradient flow in parameter space and linear interpolation in output space. This equivalence implies geodesic-like straight-line convergence to a global optimum in output space. Contribution/Results: This work provides the first mathematically precise characterization linking gradient flow dynamics directly to linear interpolation in output space, transcending conventional parameter-space-centric analysis. Leveraging tools from differential geometry and nonlinear mapping theory, it offers an interpretable geometric perspective on optimization trajectories grounded in output-space structure. Crucially, the framework guarantees reachability of global minima under mild regularity conditions, thereby unifying dynamical systems analysis with geometric optimization principles.

Differentiable optimization in robot visual localization often suffers from local minima and gradient distortion—especially in low-light keypoint detection—compromising robustness and accuracy. Method: This paper proposes a certifiably differentiable framework based on polynomial optimization (POP), which reformulates POP problems into semidefinite programming (SDP) relaxations with certified backward propagation. It integrates implicit differentiation with PyTorch-based end-to-end training, ensuring global optimality guarantees while maintaining computational efficiency. Contribution/Results: To the best of our knowledge, this is the first work to enable certified backpropagation through SDP relaxations of POP problems, theoretically guaranteeing gradient correctness. Experiments demonstrate that the method substantially mitigates failure modes of mainstream differentiable optimizers, significantly improving both keypoint detection robustness and localization accuracy under low-light conditions in robotic visual localization tasks.

This work addresses the challenge of integrating hard constraints in deep learning, where conventional orthogonal projections onto constraint sets often induce gradient saturation and impede optimization. To overcome this limitation, the authors propose a differentiable soft radial projection layer that maps inputs from Euclidean space radially into the interior of the feasible set, thereby guaranteeing strict feasibility while avoiding vanishing gradients. The method features a Jacobian matrix that is full-rank almost everywhere, effectively circumventing the gradient degeneracy associated with traditional boundary-based projections, and preserves the universal approximation capability of neural networks. By combining differentiable reparameterization with end-to-end training, the proposed approach consistently outperforms state-of-the-art optimization and projection baselines in both convergence speed and solution quality.

This work addresses the inefficiency and numerical instability of existing trajectory optimization methods, which typically rely on numerical or automatic differentiation to compute Jacobians of high-order time derivatives—such as jerk and rate of force change—while neglecting the structural properties of multibody systems. The authors propose a novel analytical framework that explicitly models physical quantities and their higher-order derivatives by leveraging the inherent structure of multibody dynamics. For the first time, they derive structured Jacobian matrices with respect to generalized coordinates and their higher-order time derivatives. By integrating analytical differentiation with multibody dynamics, the method enables efficient and scalable forward and inverse optimization. It significantly improves computational efficiency and numerical stability compared to conventional approaches and demonstrates success in accurately recovering cost function weights from motion data in inverse optimization tasks.

This work addresses the challenge of end-to-end training in deep neural networks with linearly constrained projection layers, where nonsmoothness hinders backpropagation due to the lack of theoretical guarantees and efficient algorithms. The authors propose an efficiently computable HS-Jacobian and establish, for the first time, that it constitutes a conservative mapping for polyhedral projection operators. This Jacobian is seamlessly integrated into a nonsmooth automatic differentiation framework, enabling standard optimizers such as Adam to perform end-to-end training. The resulting method provides the first training scheme with convergence guarantees for neural networks incorporating linear constraints, achieving substantial performance gains over existing approaches across diverse applications—including finance, computer vision, and neural architecture design—while maintaining both theoretical rigor and computational efficiency.

This work addresses optimal control of infinite-dimensional systems governed by partial differential equations (PDEs). We propose an end-to-end differentiable learning framework that jointly achieves reference tracking, satisfaction of state and control constraints, and minimization of control curvature. Methodologically, we employ a time-integration DeepONet (TI-DeepONet) as a high-fidelity, differentiable PDE surrogate model; integrate numerical derivative learning with automatic differentiation; and embed the surrogate into a differentiable model predictive control architecture—thereby preserving temporal causality while mitigating error accumulation in long-horizon predictions. The control policy is obtained via offline optimization of the expected control loss, eliminating the need for online optimization or supervised labels. Experiments on the heat equation, Burgers equation, and reaction-diffusion equation demonstrate that the learned policy exhibits strong generalization across diverse initial conditions and PDE parameter distributions, significantly outperforming baseline methods.

This work addresses the challenges of stability degradation and accuracy loss in neural differential equations during long-term integration, as well as the high computational cost associated with training on long trajectories. To overcome these issues, the authors propose two low-cost Jacobian regularization strategies: one that directly computes directional derivatives when the dynamics are known, and another that employs finite-difference approximations when they are not. These approaches significantly enhance the long-term simulation stability of models trained on short trajectories while substantially reducing training overhead. The method demonstrates robustness and scalability across multiple ordinary and partial differential equation systems, offering an efficient pathway for learning large-scale dynamical systems.