š¤ AI Summary
This work addresses the challenges in high-dimensional discrete-time dynamic programming where recursive utility lacks a closed-form expression and the certainty equivalent in the Bellman equation is computationally intractable. The paper proposes a Certainty Equivalent Learning (CEL) algorithm that, for the first time, integrates deep learning into the recursive utility framework. By jointly approximating the value function, policy function, and certainty equivalent function with neural networks, CEL operates without grids, Euler equations, or differentiability assumptions on the state transition dynamics. As a purely simulation-based, mesh-free solver, the method achieves high-accuracy approximations across several high-dimensional economic and financial models, yielding out-of-sample Bellman errors and first-order condition residuals on the order of 1eā»ā“ to 1eā»Ā³.
š Abstract
We propose the first deep learning algorithm, the Certainty Equivalent Learning (CEL) algorithm, for solving high-dimensional discrete-time dynamic programming problems with recursive utility. Dynamic programming with recursive utility is numerically challenging because the recursive utility does not have an explicit representation and the Bellman equation contains a certainty equivalent that is difficult to evaluate. The CEL algorithm learns this certainty-equivalent value directly with neural networks and jointly approximates value functions, policy functions, and certainty-equivalent functions. The CEL algorithm is mesh-free and simulation-based, allowing high-dimensional state and control spaces, and does not rely on Euler equations, first-order conditions, or differentiability of the state transition function. The CEL algorithm also works for dynamic programming problems with expected utility as expected utility is a special case of recursive utility. We apply the CEL to discounted linear exponential quadratic Gaussian control, small-noise robust control, Epstein-Zin DSGE, and multivariate strategic asset allocation problems. Compared with closed-form and VFI-based benchmarks, the CEL delivers accurate value and policy approximations, remains effective in high-dimensional problems, achieves accuracy comparable to VFI in the small-noise robust-control case, and produces out-of-sample Bellman errors and Euler or first-order residuals that are in the range from 1.0e-4 to 1.0e-3 for most problems.