A Policy Iteration Method for Inverse Mean Field Games

2024-09-10

arXiv.org

AI Summary

This work addresses the inverse problem of mean-field games (MFGs): reconstructing the obstacle function from partially observed value functions. To overcome the computational intractability of solving the coupled nonlinear forward–backward PDE system inherent in the forward MFG, we propose— for the first time—a decoupled framework based on policy iteration. Our method alternates between solving a linear PDE and a regularized linear inverse problem, inheriting the semantic structure of fixed-point iteration and provably achieving linear convergence. Numerical experiments in 1D and 2D using finite-difference discretization demonstrate that our approach significantly outperforms direct least-squares methods in accuracy, computational efficiency, robustness to observation noise, and scalability. To the best of our knowledge, this is the first inverse MFG solver that simultaneously offers rigorous theoretical guarantees and practical efficacy.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Multiagent Systems: Mechanism Design
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We propose a policy iteration method to solve an inverse problem for a mean-field game (MFG) model, specifically to reconstruct the obstacle function in the game from the partial observation data of value functions, which represent the optimal costs for agents. The proposed approach decouples this complex inverse problem, which is an optimization problem constrained by a coupled nonlinear forward and backward PDE system in the MFG, into several iterations of solving linear PDEs and linear inverse problems. This method can also be viewed as a fixed-point iteration that simultaneously solves the MFG system and inversion. We prove its linear rate of convergence. In addition, numerical examples in 1D and 2D, along with performance comparisons to a direct least-squares method, demonstrate the superior efficiency and accuracy of the proposed method for solving inverse MFGs.

Problem

• Reconstruct obstacle function in MFG from partial observations
• Decouple inverse MFG into linear PDEs and inverse problems
• Prove linear convergence and demonstrate superior efficiency

Innovation

• Policy iteration for inverse mean-field games
• Decouples nonlinear PDEs into linear steps
• Proves linear convergence with numerical validation