ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

2026-01-19

AI Summary

This work proposes ChartAttack, a framework that systematically uncovers and quantifies the security risks posed by multimodal large language models (MLLMs) in chart generation, where adversarial misuse can produce misleading visualizations that induce erroneous data interpretations. By integrating adversarial prompt engineering with principles of visual design, ChartAttack establishes a scalable pipeline for automated generation and evaluation of deceptive charts. The study also introduces AttackViz, a new chart question-answering dataset to support this analysis. Experimental results demonstrate significant performance degradation: MLLM-based question answering accuracy drops by an average of 19.6 and 14.9 percentage points under in-domain and cross-domain settings, respectively. Human subject evaluations further reveal a 20.2 percentage point decline in interpretation accuracy, confirming the real-world impact of this emerging threat.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, enabling efficient data analysis and reporting but also introducing new misuse risks. In this work, we introduce ChartAttack, a novel framework for evaluating how MLLMs can be misused to generate misleading charts at scale. ChartAttack injects misleaders into chart designs, aiming to induce incorrect interpretations of the underlying data. Furthermore, we create AttackViz, a chart question-answering (QA) dataset where each (chart specification, QA) pair is labeled with effective misleaders and their induced incorrect answers. Experiments in in-domain and cross-domain settings show that ChartAttack significantly degrades the QA performance of MLLM readers, reducing accuracy by an average of 19.6 points and 14.9 points, respectively. A human study further shows an average 20.2 point drop in accuracy for participants exposed to misleading charts generated by ChartAttack. Our findings highlight an urgent need for robustness and security considerations in the design, evaluation, and deployment of MLLM-based chart generation systems. We make our code and data publicly available.

Problem

• multimodal large language models
• chart generation
• misleading charts
• malicious prompting
• data misinterpretation

Innovation

• ChartAttack
• multimodal LLMs
• misleading visualization
• adversarial prompting
• chart QA