Learning to Perceive the World Through Control: Empowerment-Based Representation Learning

2026-05-28

AI Summary

This work investigates how to learn environment representations from high-dimensional observations that retain only control-relevant features. To this end, it proposes a representation learning method grounded in the empowerment objective, which maximizes an agent’s influence over its environment to jointly learn forward and backward state representations invariant to control-irrelevant variables through active interaction. This approach implicitly constructs a control-centric world model and demonstrates that control-driven interaction naturally induces complementary and invariant representations, highlighting the pivotal role of interaction in unsupervised representation learning. Experimental results show that the learned representations effectively extract control-relevant factors from high-dimensional observations, exhibiting strong invariance and utility for downstream tasks, thereby validating the superiority of control-oriented objectives in representation learning.

Abstract

In many practical reinforcement learning environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-relevant features of the environment? We study this question through the empowerment objective, which maximizes an agent's influence over the environment and is widely used for unsupervised skill learning. We show that empowerment agents induce two distinct representations -- forward and backward -- that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. Thus, empowerment maximization leads agents to learn an implicit, control-centric model of the world. Our analysis highlights the importance of learning representations through interaction rather than from passive datasets: interaction aimed at maximizing control is essential for learning useful invariance properties, a perspective that aligns closely with the causal learning literature.

Problem

• representation learning
• control-relevant features
• empowerment
• invariance
• reinforcement learning

Innovation

• empowerment
• representation learning
• control-relevant features
• invariance
• interactive learning