Artifacts as Memory Beyond the Agent Boundary

2026-04-09

AI Summary

This study investigates how leveraging environmental information can reduce an agent’s reliance on internal memory to enhance decision-making efficiency. Inspired by embodied cognition, the work formalizes the environment’s role as an external memory substrate within a reinforcement learning framework, introducing the concept of “artifacts” and providing the first mathematical characterization of the environment’s capacity to compress historical information through observations. Theoretical analysis and empirical results demonstrate that path-related information implicitly embedded in perceptual streams naturally diminishes the internal memory requirements for policy learning, and that relying solely on the observation space can substantially improve policy performance. This work reveals the inherent potential of environmental structure to serve as an efficient memory mechanism.

Technology Category

• Machine Learning: Reinforcement Learning
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The situated view of cognition holds that intelligent behavior depends not only on internal memory, but on an agent's active use of environmental resources. Here, we begin formalizing this intuition within Reinforcement Learning (RL). We introduce a mathematical framing for how the environment can functionally serve as an agent's memory, and prove that certain observations, which we call artifacts, can reduce the information needed to represent history. We corroborate our theory with experiments showing that when agents observe spatial paths, the amount of memory required to learn a performant policy is reduced. Interestingly, this effect arises unintentionally, and implicitly through the agent's sensory stream. We discuss the implications of our findings, and show they satisfy qualitative properties previously used to ground accounts of external memory. Moving forward, we anticipate further work on this subject could reveal principled ways to exploit the environment as a substitute for explicit internal memory.

Problem

• artifacts
• external memory
• reinforcement learning
• situated cognition
• memory reduction

Innovation

• artifacts
• external memory
• reinforcement learning
• situated cognition
• memory reduction