automatic differentiation

SciPy’s `spatial.transform` module currently supports only NumPy, hindering its integration into GPU-accelerated and differentiable machine learning workflows. Method: We comprehensively refactor the module to achieve full backend interoperability with JAX, PyTorch, and CuPy—while adhering to the Python Array API Standard—enabling hardware acceleration (GPU/TPU), automatic differentiation, and JIT compilation for 3D rigid-body transformations (rotations and translations). The redesign preserves NumPy-like APIs, ensures vectorized batch processing, and supports high-precision arithmetic. Contribution/Results: The restructured module has been merged into SciPy’s main branch and will be released in the upcoming version. It significantly enhances reliability, computational efficiency, and end-to-end differentiability of spatial transformations in differentiable systems—including robotics, computer vision, and physics simulation—without compromising usability or numerical robustness.

This work addresses the prohibitive computational cost of differentiable physics solvers in neural network training, caused by excessive iterative steps. We propose a progressive refinement strategy: employing coarse-grained physical solving during early training stages—first theoretically proving its sufficiency for full-precision model convergence—and dynamically controlling refinement via an adaptive termination mechanism that preserves accuracy while substantially reducing cost. Methodologically, we integrate bilevel optimization with implicit and unrolled differentiation to enable differentiable, fine-grained control over iterative linear solvers for sparse discrete differential operators. Evaluated on neural surrogate modeling of the Navier–Stokes equations, our approach reduces training time by 62% while maintaining identical inversion accuracy and generalization performance.

This work addresses the longstanding absence of second-order derivatives (Hessians) for implicit functions in differentiable finite element physics. We present the first systematic derivation and implementation of an implicit Hessian-vector product algorithm based on primitive automatic differentiation operators, enabling PDE-constrained optimization. Our method leverages Jacobian-vector and vector-Jacobian products as core primitives, enabling seamless integration of Newton-CG and L-BFGS-B optimizers into finite element solvers. We establish a verifiable and scalable second-order implicit differentiation paradigm, thereby bridging a critical theoretical and practical gap in differentiable physics concerning Hessian computation. The approach is validated on four 2D/3D benchmark problems—spanning both linear and nonlinear regimes—with verified numerical accuracy. When combined with exact Hessians, Newton-CG accelerates convergence by 2–5× for traction identification and shape optimization tasks.

Aligning highly deformable object simulations with real-world behavior remains challenging due to the difficulty of estimating underlying physical parameters from sparse, noisy observations. Method: This paper proposes an end-to-end differentiable simulation-rendering closed-loop framework. It integrates differentiable point cloud sampling with differentiable physics simulation (DiffSim) to directly invert physical parameters—such as mass and stiffness—from real point cloud observations. Coupled with differentiable point cloud rendering and neural architecture search, the framework establishes a fully differentiable optimization pathway where gradients flow back to physical parameters. Results: Experiments on diverse soft-body objects demonstrate that the method matches or surpasses manual parameter tuning in accuracy, achieves over 10× faster parameter convergence, and significantly accelerates simulation modeling and deployment for novel tasks.

Existing physics engines for robotics struggle to simultaneously ensure stable simulation, high-fidelity rigid contact modeling, and full differentiability with respect to states, actions, and system parameters. To address this, we propose Dojo—the first end-to-end differentiable physics engine designed specifically for robotics. Dojo uniquely integrates variational integrators with a second-order cone nonlinear complementarity problem (NCP) solver, guaranteeing energy and momentum conservation during contact and enabling smooth, analytic gradient computation across contact events. It further employs a customized primal-dual interior-point method for efficient implicit differentiation. Evaluated on motion planning, policy optimization, and system identification tasks, Dojo demonstrates significantly improved gradient accuracy and faster optimization convergence in challenging rigid-contact scenarios.

Legacy scientific codebases, such as those written in Fortran, pose significant barriers to integration with modern differentiable frameworks, thereby limiting the application of gradient-based methods in parameter estimation and data assimilation. This work proposes a large language model (LLM)-driven, five-stage agent pipeline that combines static dependency analysis, iterative compilation repair, Fortran reference oracle validation, and JAX automatic differentiation to achieve the first fully automated, high-fidelity, differentiable translation of a large-scale Earth system model component—specifically, the 19,000-line Fortran land surface model CLM-ml-v2. The resulting framework substantially lowers the barrier to modernizing scientific computing models: it enables computation of the full Jacobian via a single backward pass, accelerates parameter inversion by 8× compared to gradient-free optimization, and achieves a 24× speedup over the original Fortran implementation when scaling to 2,048 parallel samples.

Existing differentiable PDE solvers lack a unified benchmark to evaluate their practical performance in terms of gradient correctness, computational overhead, numerical stability, and ease of integration. This work proposes an extensible benchmarking framework that, through containerized encapsulation (Tesseract) and a standardized gradient API, supports cross-language interoperability and multiple automatic differentiation strategies. For the first time, it enables a systematic comparison of 14 differentiable solvers spanning fluid dynamics, structural mechanics, and heat transfer. Experiments reveal order-of-magnitude differences in computational cost and Jacobian condition numbers across solvers, yet all converge to similar optimal solutions. These findings indicate that real-world bottlenecks lie primarily in memory consumption, numerical stability, and compatibility—not in optimization capability.

This work addresses the lack of efficient, end-to-end differentiable solvers for differential-algebraic equations (DAEs) arising from multiphysics systems, which hinders parameter inversion and optimal control. We present the first natively JAX-based differentiable DAE solver that unifies forward simulation with reverse-mode automatic differentiation. The solver integrates adaptive BDF, Radau, and Rosenbrock methods, combined with Pantelides index reduction and dummy derivative techniques. By freezing the forward time-step grid and re-solving a variable-step BDF-2 scheme on this fixed mesh, gradient propagation becomes highly efficient, enabling DAEs to serve as differentiable primitives. A single call to jax.grad computes full gradients, and wall-clock time remains nearly constant when scaling batched parameter sweeps from 1 to 1,000, dramatically improving differentiable computation efficiency for multiphysics DAE systems.

This work addresses the longstanding divide between program execution and gradient-based optimization, which has hindered the use of general-purpose code as learnable scientific models. The authors propose the Differentiable Meta-Circular Interpreter (DMCI), which compiles a self-hosted subset of Scheme into a differentiable computation graph, enabling exact reverse-mode automatic differentiation of arbitrary recursive, higher-order programs with closures and complex data structures—without requiring recompilation or custom gradient definitions. This approach facilitates joint optimization over both program structure and parameters. Validated on 171 programs for gradient correctness, DMCI significantly outperforms handcrafted models and gradient-free methods in inverse problems such as battery degradation and El Niño prediction, successfully achieving symbolic regression and extrapolation with stateful executable programs.

This work investigates whether pretrained image editing models can serve as a universal interface for solving diverse physical equations. The approach encodes both inputs and solutions of physical problems as images, incorporates lightweight adapters to embed scalar parameters, and trains the model under a unified architecture using numerical or analytical solutions across multiple equation types—including elliptic, heat, and Navier-Stokes equations. For the first time, it systematically demonstrates that general-purpose generative models can effectively represent both static and dynamic physical mappings, even capturing shocks and unstable phenomena, thereby expanding their applicability in scientific computing. Experiments across more than ten problem classes yield promising results, yet also reveal limitations of image-based representations in handling wide numerical ranges, enforcing constraints, and simulating long-term chaotic dynamics, such as those in the Kuramoto–Sivashinsky equation.