Mutual Information Optimal Control of Discrete-Time Linear Systems

2025-07-07

AI Summary

This paper addresses the mutual information optimal control problem (MIOCP) for discrete-time linear systems, aiming to jointly optimize both the control policy and the prior distribution—thereby relaxing the restrictive assumption in maximum entropy optimal control (MEOCP) that the prior is fixed as uniform. Under Gaussian policy and Gaussian prior assumptions, we derive closed-form analytical solutions for both the policy and the prior via variational inference, and propose an alternating minimization algorithm with provable convergence. To the best of our knowledge, this is the first work to incorporate mutual information as the control objective within the discrete linear system framework, significantly enhancing policy expressiveness and improving the exploration-exploitation trade-off. Numerical experiments demonstrate superior control performance and robustness compared to baseline methods, establishing a novel information-theoretic paradigm for optimal control.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

In this paper, we formulate a mutual information optimal control problem (MIOCP) for discrete-time linear systems. This problem can be regarded as an extension of a maximum entropy optimal control problem (MEOCP). Differently from the MEOCP where the prior is fixed to the uniform distribution, the MIOCP optimizes the policy and prior simultaneously. As analytical results, under the policy and prior classes consisting of Gaussian distributions, we derive the optimal policy and prior of the MIOCP with the prior and policy fixed, respectively. Using the results, we propose an alternating minimization algorithm for the MIOCP. Through numerical experiments, we discuss how our proposed algorithm works.

Problem

• Extends maximum entropy control to optimize policy and prior jointly
• Derives optimal Gaussian policy and prior for mutual information control
• Proposes alternating minimization algorithm for mutual information optimization

Innovation

• Mutual information optimizes policy and prior
• Gaussian distributions used for optimal solutions
• Alternating minimization algorithm proposed