An Information-Theoretic Definition for Open-Ended Learning

2026-06-06

AI Summary

This study addresses the lack of a rigorous definition of open-ended learning and a theoretical framework enabling agents to continuously explore in open environments. The authors formally define open-ended learning through an information-theoretic lens and introduce a novel “bit-equivalence” metric to quantify the amount of information required to achieve varying levels of reward, thereby characterizing the openness of an environment. Their key contributions include proving that classical multi-armed bandit settings are inherently non-open-ended, constructing the first multi-armed bandit environment that satisfies the proposed openness criteria, and designing a new exploration algorithm that enables continual growth of agent capabilities within this environment.

Abstract

A growing body of work points to the great promise of AI systems that can continually expand their capabilities as they operate in an open-ended environment. But yet there is no coherent definition of open-endedness or theory about how an agent ought to explore an open-ended environment. We introduce an information-theoretic definition based on a new concept -- the ${\textit bit-equivalent}$ -- which quantifies the information required to attain each level of expected reward. We consider an environment to be open-ended if an agent can attain linear growth in the bit-equivalent. We establish that classical bandit environments are not open-ended and formulate a bandit environment that is. We also introduce an algorithm that achieves open-ended learning in this environment.

Problem

• open-ended learning
• open-endedness
• information-theoretic definition
• bit-equivalent
• AI systems

Innovation

• open-ended learning
• information-theoretic
• bit-equivalent
• bandit environment
• exploration