Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance

2026-07-28

AI Summary

This work addresses the challenge of effectively leveraging privileged state information to improve observation representation learning in model-based reinforcement learning, particularly under asymmetric observation settings. Building upon the Dreamer framework, the authors propose a novel asymmetric world model training approach that introduces a latent guidance mechanism and a lightweight asymmetric representation learning objective. This design enhances the model’s capacity to exploit privileged information without requiring complex architectural modifications. Experimental results demonstrate that the proposed method consistently outperforms both the original Dreamer and existing asymmetric approaches across multiple benchmark tasks, achieving significant and stable performance gains.

Abstract

Much like humans benefit from guidance while learning, reinforcement learning algorithms may benefit from additional supervision beyond rewards. Leveraging additional information during training to learn better representations and behaviors has been the focus of asymmetric reinforcement learning. This learning paradigm has proven effective under partial observability when additional state information is available, but also under full observability when more refined state information is available. Focusing on model-based reinforcement learning, we study the effect of asymmetric learning on observation representations and on privileged information representations. First, we identify a limitation in the privileged information representations learned by an asymmetric model-based algorithm known as the Informed Dreamer. Then, we propose a novel asymmetric representation learning objective using latent guidance, resulting in a new algorithm called the Reinformed Dreamer. Experiments across several benchmarks show a more consistent improvement over Dreamer than previous asymmetric approaches.

Problem

• asymmetric reinforcement learning
• model-based reinforcement learning
• privileged information
• representation learning
• latent guidance

Innovation

• asymmetric reinforcement learning
• latent guidance
• world model
• model-based reinforcement learning
• privileged information