MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction

2026-07-22

AI Summary

This work addresses the lack of evaluation frameworks for assessing multimodal large language models’ ability to jointly reason about data semantics, view coordination, and interaction logic in collaborative multi-view interface construction. The authors propose MV-Bench, the first task-specific benchmark built upon real-world Tableau workbooks, which leverages a structured intermediate representation and an executable web interface generation pipeline. MV-Bench encompasses 92 base interfaces and 1,048 validation instances, and introduces an automated evaluation along three dimensions: visual fidelity, data-binding correctness, and interaction completeness. While the strongest model achieves 75.45% accuracy in layout reproduction, its performance drops significantly in data binding (21.71%) and interaction completeness (11.68%), revealing critical bottlenecks in current models’ capacity for semantic understanding and interactive behavior generation.

Abstract

Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations mainly focus on single-chart generation and overlook coordinated multi-view interface construction, which requires joint reasoning about data semantics, view coordination, and interaction logic. Consequently, MLLM capabilities in this setting remain underexplored, and the field lacks a dedicated benchmark for systematic assessment. We introduce MV-Bench, a benchmark for evaluating MLLMs on coordinated multi-view interface construction. Instead of relying on incomplete or inconsistent open-source implementations, we use Tableau workbook files as ground truth because they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web interfaces through structured intermediate representations. The benchmark contains 92 base interfaces and 1,048 verified instances created by recombining chart types, datasets, and interaction patterns. Each instance includes executable code, a rendered interface, a dataset, and interaction annotations. We evaluate five state-of-the-art MLLMs in a single-pass setting using metrics for visual fidelity, data binding correctness, and interaction completeness. The strongest model achieves 75.45 percent accuracy in visual layout reproduction, but only 21.71 percent in data binding and 11.68 percent in interaction completeness. These results show that current MLLMs can reproduce visual appearance but remain limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Iterative refinement improves code executability but does not substantially reduce the gap in data binding and interaction generation.

Problem

• multimodal large language models
• coordinated multi-view interfaces
• visualization generation
• benchmarking
• interaction logic

Innovation

• coordinated multi-view
• multimodal large language models
• visualization benchmark
• data binding
• interaction logic