SynGallery: A Synthetic Gallery of Real Paintings for Instance-Level Artwork Recognition

2026-07-21

AI Summary

This work addresses the domain gap in instance-level artwork identification between clean, curated training images and real-world query photographs, which often suffer from variations in viewpoint, lighting, and reflections. To bridge this gap, the authors propose SynGallery, a method that embeds museum-curated paintings into procedurally generated 3D gallery scenes to render large-scale synthetic training data under diverse viewpoints and appearance conditions. This approach is the first to leverage geometrically and photometrically controllable synthetic data to mitigate domain discrepancies, revealing the critical role of viewpoint diversity in performance gains. Training solely on SynGallery improves the GAP⁻ metric from 67.18 to 73.47, and when combined with the original Met dataset, it elevates benchmark performance from 35.97 to 38.48.

Abstract

Instance-level artwork recognition requires matching a handheld visitor photograph to a specific work in a large museum collection. This is challenging because painting datasets typically provide clean catalog images for training, while test queries are captured under oblique viewpoints, gallery lighting, reflections, frames, and other scene-level variations. We present SynGallery, a synthetic gallery dataset for artwork retrieval that addresses this gap without collecting additional real photographs. Starting from catalog images of real paintings, we place each artwork into a procedurally generated 3D gallery scene and render it from multiple viewpoints under varied geometric and appearance conditions, while preserving the exact identity of the original work. The resulting dataset contains 24,490 rendered views of 4,898 paintings from the Met benchmark. We show that these synthetic views provide a stronger training signal than the corresponding studio photographs. At the same number of training data points, training only on SynGallery improves art painting recognition from 67.18 to 73.47 GAP$^-$. When added to the full Met training set, SynGallery improves the published benchmark protocol from 35.97 to 38.48 GAP. Ablation experiments show that the gain comes primarily from geometric viewpoint variation rather than photographic realism: blur, sensor noise, and image compression consistently reduce performance.

Problem

• instance-level artwork recognition
• viewpoint variation
• gallery conditions
• domain gap
• artwork retrieval

Innovation

• SynGallery
• instance-level artwork recognition
• synthetic dataset
• 3D gallery rendering
• domain gap