On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians

2026-01-31

AI Summary

This work addresses the challenge of lacking explicit optimal control laws in optimal feedback control problems involving implicit Hamiltonians by proposing a stochastic minibatch optimization method based on Jacobian-Free Backpropagation (JFB). It provides, for the first time, theoretical guarantees for JFB’s convergence to desired stationary points under stochastic settings and extends this framework to high-dimensional multi-agent systems. By integrating implicit deep learning with optimal control theory, the approach demonstrates strong scalability and effectiveness across a range of high-dimensional tasks, including multi-agent consumption, quadrotor swarms, and bicycle control.

Technology Category

• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Optimal feedback control with implicit Hamiltonians poses a fundamental challenge for learning-based value function methods due to the absence of closed-form optimal control laws. Recent work~\cite{gelphman2025end} introduced an implicit deep learning approach using Jacobian-Free Backpropagation (JFB) to address this setting, but only established sample-wise descent guarantees. In this paper, we establish convergence guarantees for JFB in the stochastic minibatch setting, showing that the resulting updates converge to stationary points of the expected optimal control objective. We further demonstrate scalability on substantially higher-dimensional problems, including multi-agent optimal consumption and swarm-based quadrotor and bicycle control. Together, our results provide both theoretical justification and empirical evidence for using JFB in high-dimensional optimal control with implicit Hamiltonians.

Problem

• optimal control
• implicit Hamiltonians
• value function methods
• closed-form control laws
• feedback control

Innovation

• Jacobian-Free Backpropagation
• implicit Hamiltonians
• optimal control
• convergence guarantees
• stochastic minibatch