ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning

2026-08-04

AI Summary

This work addresses the challenges of low sample efficiency, insufficient robustness under non-stationary environments, and catastrophic forgetting in reinforcement learning by proposing a structured disentanglement approach that decouples transition dynamics from reward signals. The proposed architecture integrates Grow-When-Required networks with Successor Features, enabling rapid adaptation to new goals and facilitating forward and backward knowledge transfer to effectively mitigate forgetting. Evaluated on non-stationary spatial navigation tasks, the model substantially outperforms state-of-the-art algorithms, achieving near-instantaneous goal adaptation and markedly improved sample efficiency.

Abstract

Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficiency, but still fail when the environment shifts. This paper introduces Adaptive Topological Learning with Abstract Successors (ATLAS) to combat these challenges. ATLAS uses a Grow When Required network with Successor Features in order to achieve high sample efficiency while also robustly tackling catastrophic forgetting. We evaluate ATLAS in spatial navigation tasks, benchmarking its performance against common on-policy and off-policy algorithms. Our empirical results demonstrate that by structurally decoupling transition dynamics from the reward signal, ATLAS achieves near-instantaneous adaptation to new goals and can exhibit positive backward transfer, significantly outperforming baseline methods in non-stationary environments.

Problem

• continual learning
• catastrophic forgetting
• sample efficiency
• non-stationary environments
• reinforcement learning

Innovation

• Adaptive Topological Learning
• Successor Features
• Grow When Required network
• Catastrophic Forgetting
• Non-stationary Environments