design differentiable models

In engineering design, the non-differentiability of conventional CAE workflows—particularly mesh generation and physics simulation—hinders gradient-based high-dimensional optimization. To address this, we propose an end-to-end differentiable shape optimization framework: geometry is represented via signed distance fields (SDFs), and a 3D U-Net serves as a full-field surrogate model that directly learns the mapping from SDFs to physical fields (e.g., pressure, velocity), thereby bypassing non-differentiable components without requiring differentiable solvers or adjoint methods. The surrogate is embedded within a differentiable optimization pipeline, enabling backpropagation to compute gradients with respect to design parameters. Evaluated on aerodynamic shape optimization, our method achieves fully gradient-driven, efficient iterative design refinement. Results demonstrate substantial improvements in optimization efficiency and validate the framework’s feasibility and advantages in complex engineering applications.

Existing differentiable optimization frameworks suffer from fragmented modeling interfaces, cumbersome parameter differentiation, and poor solver compatibility. This paper introduces the first general-purpose, differentiable optimization framework natively supporting parametric JuMP modeling. Grounded in the Karush–Kuhn–Tucker (KKT) conditions and under standard regularity assumptions, it unifies forward- and reverse-mode sensitivity analysis for both convex and nonconvex problems. Key contributions include: (1) first-class parameter abstractions enabling automatic, named-parameter differentiation across objectives and constraints—eliminating low-level coefficient manipulation; (2) deep integration with DiffOpt.jl and the JuMP ecosystem while preserving solver agnosticism; and (3) empirical validation on economic dispatch, portfolio optimization, and robotic inverse kinematics, plus successful deployment in energy market bidding and end-to-end Sobolev training—demonstrating substantial improvements in the modeling–optimization–learning feedback loop efficiency.

This work addresses the efficient and robust computation of gradients for numerical solutions of differential equations. We systematically survey four differentiable programming paradigms—adjoint methods, automatic differentiation (via source-to-source transformation and operator overloading), numerical perturbation, and symbolic-numeric hybrid approaches—and introduce, for the first time, a unified differentiability framework that bridges inverse problem solving and machine learning methodologies. We establish a cross-method comparative taxonomy and provide platform-specific best-practice guidelines for scientific computing libraries including SciPy, JAX, and TorchDiffeq. Our analysis rigorously characterizes trade-offs among accuracy, memory footprint, computational complexity, and applicability domains for each method. The results deliver both theoretical foundations and practical implementation pathways for differential-equation–data fusion modeling tasks, including parameter inversion, sensitivity analysis, and physics-informed neural networks (PINNs).

This paper addresses the challenge of end-to-end differentiability in complex programs featuring nontrivial control flow and data structures. To this end, it introduces a probabilistic programming paradigm for differentiation, unifying optimization and probabilistic inference within a differentiable programming framework. Methodologically, it transcends conventional automatic differentiation (AD) by establishing, for the first time, a theoretical link between differentiability of control flow/data structures and uncertainty modeling—integrating AD, graphical models, convex optimization, and Bayesian inference into a cohesive differentiable program modeling framework. Key contributions include: (1) revealing that differentiable programming is fundamentally probabilistic programming—not merely gradient computation; (2) proposing the “program-as-model” design principle; and (3) establishing the first comprehensive knowledge system spanning theory, design, and applications, enabling the development of differentiable software infrastructure for large language models and foundation models.

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

This work addresses the challenges in inverse material design posed by discrete parameters, physical constraints, and solution multiplicity, which often render gradient-based optimization ineffective. To overcome these limitations, the authors propose a differentiable framework that integrates continuous relaxation with guided diffusion. By relaxing the discrete design space into a continuous grid and combining differentiable finite element simulation with implicit differentiation, the method leverages a diffusion model during inference, steered by a multi-objective loss function to generate physically plausible designs. This approach achieves the first integration of implicit differentiation and diffusion priors for multimodal inverse material design. It efficiently produces diverse, physically valid 2D and 3D structures with relative errors below 1%, while simultaneously optimizing multiple performance objectives such as material density.

This study addresses the PDE-constrained inverse problem of correcting Reynolds-averaged Navier–Stokes (RANS) turbulence models by introducing an end-to-end differentiable framework. The approach embeds a PDE solver into PyTorch’s automatic differentiation graph via implicit layers and incorporates a trainable additive correction term, enabling joint optimization of model parameters or neural networks. Built upon the BROADCAST solver, this work presents a unified and user-friendly differentiable PDE interface and, for the first time, achieves end-to-end differentiable correction for compressible-flow RANS models. The framework successfully optimizes the production-term coefficient in the Spalart–Allmaras model and reconstructs the eddy viscosity field in two canonical cases—NASA’s wall-mounted hump and the VKI LS-59 turbine blade—demonstrating its effectiveness and flexibility for turbulence modeling and broader physics-informed PDE inverse problems.

This work addresses the high computational cost of evaluating objective functions and their gradients, as well as slow convergence, in engineering optimization. It proposes the Learned Gradient Flow (LGF) optimizer, which employs a data-driven equation discovery approach to infer continuous-time dynamical systems from optimization trajectories—systems that correspond to algorithms such as gradient descent, Newton’s method, and ADAM. LGF constructs surrogate gradient flow models that can replace the original problem, adaptively generating polynomial surrogates of varying orders in either full-dimensional or reduced-dimensional spaces. This significantly reduces reliance on repeated evaluations of the original objective function and its gradients. Demonstrated across diverse forward and inverse problems in structural topology optimization and scientific machine learning, the method accelerates convergence while preserving essential features of the optimization trajectory.

This work addresses the challenge in Earth system deep learning where non-differentiable scientific metrics—such as area, perimeter, and connectivity—cannot be directly optimized, often yielding blurry outputs with lost high-frequency details. To overcome this, the study introduces the first differentiable loss formulation based on Minkowski functionals, achieved through temperature-controlled sigmoid relaxation and continuous logical operators to enforce geometric constraints. Furthermore, it proposes a Lipschitz convolutional network that integrates spectral normalization with hard geometric constraints, enabling high-fidelity surrogate learning of non-differentiable metrics. Evaluated on the EUMETNET OPERA dataset, the method completely eliminates geometric violations and significantly outperforms unconstrained baselines, while also uncovering an inherent trade-off between Lipschitz regularization and local texture recovery.