SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels

2024-10-11

AI Summary

This work addresses the challenges of poor interpretability and low sample efficiency in robot relational reasoning and manipulation tasks from raw pixel inputs. We propose an unsupervised object-centric latent dynamics modeling framework. Our core contribution is the first integration of Slot Attention into a model-based reinforcement learning (MBRL) architecture, enabling slot-based, disentangled latent representations that structurally decompose the environment and support interpretable dynamic modeling. The method processes raw pixels end-to-end, jointly optimizing object discovery, temporal dynamics modeling, and policy learning. Evaluated on multiple relational manipulation benchmarks, our approach significantly outperforms DreamerV3 and TD-MPC2 in both sample efficiency and cross-task generalization. These results empirically validate the effectiveness and transferability of object-centric representations for modeling complex manipulation tasks.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Learning a latent dynamics model provides a task-agnostic representation of an agent's understanding of its environment. Leveraging this knowledge for model-based reinforcement learning (RL) holds the potential to improve sample efficiency over model-free methods by learning from imagined rollouts. Furthermore, because the latent space serves as input to behavior models, the informative representations learned by the world model facilitate efficient learning of desired skills. Most existing methods rely on holistic representations of the environment's state. In contrast, humans reason about objects and their interactions, predicting how actions will affect specific parts of their surroundings. Inspired by this, we propose Slot-Attention for Object-centric Latent Dynamics (SOLD), a novel model-based RL algorithm that learns object-centric dynamics models in an unsupervised manner from pixel inputs. We demonstrate that the structured latent space not only improves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outperforms DreamerV3 and TD-MPC2 - state-of-the-art model-based RL algorithms - across a range of benchmark robotic environments that require relational reasoning and manipulation capabilities. Videos are available at https://slot-latent-dynamics.github.io/.

Problem

• Learns object-centric dynamics from pixels
• Improves sample efficiency in reinforcement learning
• Enhances interpretability and reasoning in behavior models

Innovation

• Object-centric latent dynamics
• Unsupervised learning from pixels
• Structured latent space enhances interpretability