R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentation

2026-03-18

AI Summary

This work addresses inefficiencies in image-based model-based reinforcement learning, where existing methods either waste capacity reconstructing task-irrelevant visual details or rely heavily on external data augmentation. To overcome these limitations, the authors propose R2-Dreamer, which introduces, for the first time, a Barlow Twins-inspired redundancy-reduction self-supervised objective as an internal regularization mechanism within a decoder-free framework. This approach effectively prevents representational collapse and eliminates the need for data augmentation. R2-Dreamer achieves performance on par with DreamerV3 and TD-MPC2 on the DeepMind Control Suite and Meta-World benchmarks, while training 1.59× faster. Notably, it substantially outperforms prior methods in the DMC-Subtle environment—where task-relevant objects are subtle—demonstrating superior representation efficiency and generalization capability.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Representation Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

A central challenge in image-based Model-Based Reinforcement Learning (MBRL) is to learn representations that distill essential information from irrelevant visual details. While promising, reconstruction-based methods often waste capacity on large task-irrelevant regions. Decoder-free methods instead learn robust representations by leveraging Data Augmentation (DA), but reliance on such external regularizers limits versatility. We propose R2-Dreamer, a decoder-free MBRL framework with a self-supervised objective that serves as an internal regularizer, preventing representation collapse without resorting to DA. The core of our method is a redundancy-reduction objective inspired by Barlow Twins, which can be easily integrated into existing frameworks. On DeepMind Control Suite and Meta-World, R2-Dreamer is competitive with strong baselines such as DreamerV3 and TD-MPC2 while training 1.59x faster than DreamerV3, and yields substantial gains on DMC-Subtle with tiny task-relevant objects. These results suggest that an effective internal regularizer can enable versatile, high-performance decoder-free MBRL. Code is available at https://github.com/NM512/r2dreamer.

Problem

• Model-Based Reinforcement Learning
• Representation Learning
• Decoder-Free
• Data Augmentation
• Redundancy Reduction

Innovation

• decoder-free
• redundancy reduction
• self-supervised learning
• model-based reinforcement learning
• internal regularizer