CtD: Composition through Decomposition in Emergent Communication

2026-01-15

International Conference on Learning Representations

AI Summary

This work addresses the challenge of enabling neural agents to achieve compositional generalization on unseen images within emergent communication. The authors propose a novel "composition through decomposition" paradigm: in a multi-agent coordination game, agents first learn to disentangle images into basic concepts via a learnable codebook, which is then used to compositionally generate complex descriptions. Employing a two-stage sequential training strategy, this approach achieves zero-shot compositional generalization within the emergent communication framework without requiring additional training. Experimental results demonstrate that the agents can effectively describe previously unseen images and exhibit strong compositional generalization capabilities in certain scenarios.

Technology Category

Multiagent Systems: Agent CommunicationNatural Language Processing: Code Generation / Program Synthesis from Natural LanguageComputer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactionsSearch and Retrieval-Augmented AI: Agentic searchSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Compositionality is a cognitive mechanism that allows humans to systematically combine known concepts in novel ways. This study demonstrates how artificial neural agents acquire and utilize compositional generalization to describe previously unseen images. Our method, termed"Composition through Decomposition", involves two sequential training steps. In the'Decompose'step, the agents learn to decompose an image into basic concepts using a codebook acquired during interaction in a multi-target coordination game. Subsequently, in the'Compose'step, the agents employ this codebook to describe novel images by composing basic concepts into complex phrases. Remarkably, we observe cases where generalization in the `Compose'step is achieved zero-shot, without the need for additional training.

Problem

• compositionality
• emergent communication
• compositional generalization
• neural agents
• zero-shot generalization

Innovation

• compositionality
• emergent communication
• zero-shot generalization
• neural agents
• codebook-based decomposition