Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations

2026-07-15

AI Summary

This work addresses the limitation of existing vision-language models, which compress the multidimensional relationships inherent in artworks—such as style, iconography, and historical context—into a single semantic space, thereby hindering the nuanced, multi-relational structural analysis required in art historical inquiry. To overcome this, the authors propose CANVAS, a novel framework that introduces sheaf theory into multimodal representation learning for art. CANVAS constructs distinct embedding spaces for each relational context and achieves multi-relational cross-modal alignment through context-conditioned projections and a new contrastive loss formulation. Notably, it preserves multi-relational structure at inference without relying on external data. Evaluated on three newly introduced benchmarks—WikiArt+, HertzianaDP, and SemArt+—CANVAS significantly outperforms current baselines in both cross-modal retrieval and art understanding tasks.

Abstract

Understanding a painting is never a single act. Art historians may analyze the same work through concepts of style, iconography, or historical context, dimensions that are not interchangeable, and each carries distinct semantic relationships between the visual and the textual. Vision-Language Models (VLMs) like CLIP, which learn a single shared embedding space, collapse this richness into a single homogeneous alignment, thereby losing the multi-relational structure that defines art-historical reasoning. We introduce CANVAS (Contrastive Art-aware Network for Vision-Language Alignment with Sheaves), a framework for learning relation-aware multimodal representations inspired by sheaf theory. Each artwork is projected into multiple embeddings conditioned on the type of relation (i.e., the context), and a novel contrastive loss encodes contextual information during training, with no dependency on external data at inference. We evaluate on three newly introduced benchmarks of artworks for multi-relational art understanding: WikiArt+, derived from WikiArt and Wikipedia, HertzianaDP, from the Bibliotheca Hertziana collection, and SemArt+, refined from the SemArt dataset. In multimodal retrieval and art understanding, CANVAS outperforms the baselines, supporting the view that multi-relational alignment is not just theoretically motivated but also practically essential.

Problem

• multi-relational representation
• vision-language models
• art understanding
• semantic alignment
• sheaf theory

Innovation

• sheaf theory
• contrastive learning
• multi-relational representation
• vision-language alignment
• art understanding