SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation

2026-08-03

AI Summary

Current evaluation methods for text-to-SVG generation predominantly rely on code-based or rasterized metrics, which fail to capture human perceptual judgments and overlook the geometric and spatial structural properties inherent to SVGs. To address this gap, this work proposes SVGEval—the first vision-anchored, multimodal evaluation benchmark that aligns with human judgment by integrating rendered SVG images with multi-round expert-refined human annotations across semantic, aesthetic, geometric, and layout dimensions. We introduce an interpretable, multi-aspect scoring model that uniquely leverages visual renderings as the basis for assessment and incorporates textual rationales alongside scores. Experimental results demonstrate that existing models exhibit significant deficiencies in geometric and layout fidelity, whereas the proposed evaluator substantially outperforms baseline methods on these critical dimensions.

Abstract

Multimodal large models are increasingly used to generate scalable vector graphics (SVG), but reliable evaluation remains underexplored. Existing protocols are often code-centric or borrow raster-image metrics after rendering SVGs, which fail to reflect human perception and overlook SVG-specific qualities such as geometry and spatial composition. We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment. SVGEval explicitly incorporates visual renderings to evaluate whether models can judge the rendered outcome rather than only inspect SVG code, and provides high-quality annotations obtained via multi-round human labeling with expert refinement. Systematic evaluations across representative multimodal models reveal a clear gap: models perform relatively well on semantic alignment and aesthetics, yet struggle on geometry- and layout-related judgments. Building on SVGEval, we train an explainable SVG quality scorer that outputs multi-aspect scores with textual rationales. Ablations show that explicit visual grounding and reasoning supervision are crucial, especially for spatial and geometric assessment. SVGEval offers a reliable testbed and practical scorer for evaluating and improving SVG generation in the era of multimodal models.

Problem

• SVG generation
• perceptual quality
• evaluation benchmark
• multimodal models
• human-aligned assessment

Innovation

• SVGEval
• vision-grounded evaluation
• text-to-SVG generation
• perceptual quality benchmarking
• explainable quality scorer