Successes and Limitations of Object-centric Models at Compositional Generalisation

2024-12-25

AI Summary

Existing object-centric models exhibit limited generalization to unseen attribute-level compositions (e.g., novel color–shape pairings). Method: This work presents the first systematic evaluation of object-centric models for attribute-level compositional generalization, introducing an attention-driven object decomposition mechanism, contrastive object representation learning, and a structured disentanglement training paradigm; it further establishes a dedicated evaluation protocol for compositional generalization. Contributions/Results: The proposed model significantly outperforms conventional disentanglement methods, achieving substantial gains in attribute-level compositional generalization. Ablation and analysis reveal that its efficacy stems from the disentangled and composable nature of learned object representations. Crucially, the study identifies insufficient robustness in object binding—particularly under occlusion or clutter—as a key bottleneck. Guided by this insight, we propose targeted architectural and training improvements. Overall, this work clarifies both the promise and fundamental limitations of object-centric modeling for compositional generalization.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Representation Learning
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In recent years, it has been shown empirically that standard disentangled latent variable models do not support robust compositional learning in the visual domain. Indeed, in spite of being designed with the goal of factorising datasets into their constituent factors of variations, disentangled models show extremely limited compositional generalisation capabilities. On the other hand, object-centric architectures have shown promising compositional skills, albeit these have 1) not been extensively tested and 2) experiments have been limited to scene composition -- where models must generalise to novel combinations of objects in a visual scene instead of novel combinations of object properties. In this work, we show that these compositional generalisation skills extend to this later setting. Furthermore, we present evidence pointing to the source of these skills and how they can be improved through careful training. Finally, we point to one important limitation that still exists which suggests new directions of research.

Problem

• Object-Centric Models
• Combination Learning
• Property Integration

Innovation

• object-centric models
• combinatorial learning
• methodological improvements