A Unified Framework for Empowerment and Predictive Control

2026-10-02

AI Summary

This study addresses the limitations of Model Predictive Control (MPC), which relies on manually designed cost functions, and existing empowerment-based control methods that require computationally expensive second-order derivatives while decoupling exploration from execution. To overcome these challenges, this work proposes a unified framework that integrates empowerment objectives and task behaviors within a single policy. By combining information-theoretic empowerment estimation with sampling-based MPC, the approach requires only first-order dynamics derivatives and can be optimized via standard MPC solvers, intrinsically unifying exploration and execution. This method eliminates virtual probe interference, significantly reduces computational complexity, and remains compatible with mainstream MPC paradigms. Experimental results demonstrate that the proposed approach effectively balances empowered exploration and task execution in classical control tasks, achieving success rates comparable to or exceeding those of single-objective optimization, thereby bridging standard MPC and intrinsic motivation-based control.

Abstract

Sampling-based model predictive control (MPC) is a powerful approach to trajectory optimization, but its performance depends on an informative cost function that often requires substantial domain knowledge to design. An alternative is to derive objectives directly from the system dynamics. Empowerment, defined as the channel capacity between an agent's inputs and its future state, provides such a goal-agnostic objective and has been shown to produce useful behaviors across a range of domains. However, existing empowerment-based controllers have two key limitations: empowerment is estimated using a virtual probing policy distinct from the executed control policy, obscuring its connection to the resulting behavior; and its computation typically requires second-order derivatives of the dynamics, limiting compatibility with standard MPC methods. We formulate empowerment using a single policy for both probing and acting, directly linking the empowerment objective to the executed behavior. The resulting objective requires only first-order derivatives and can be optimized with standard MPC solvers, either alone or in combination with a task-specific cost. We evaluate the proposed approach on classical control tasks and show that combining empowerment with task costs matches or exceeds the success rate achieved by either objective alone. Our formulation provides a practical bridge between standard MPC and intrinsically motivated control.

Problem

• Model Predictive Control
• Empowerment
• Trajectory Optimization
• Intrinsic Motivation

Innovation

• Empowerment
• Model Predictive Control
• Intrinsic Motivation
• Trajectory Optimization
• Unified Framework