Touching or Chatting: The Utility of LLMs and Tactile Charts for Learning about Complex Chart Types by BLV Individuals

2026-07-25

AI Summary

This study addresses the challenges blind and low-vision (BLV) users face in constructing spatial mental models when learning complex charts, which hinders their comprehension of visualizations and collaboration with sighted peers. For the first time, it systematically evaluates the impact of integrating tactile charts with large language models (LLMs) on BLV users’ chart learning. A controlled experiment with 12 BLV participants compared conditions combining tactile graphics, alternative text, and an LLM-powered chatbot against text-only inputs with LLM support. Results demonstrate that the tactile + text + LLM condition significantly outperforms text + LLM alone, particularly in tasks requiring spatial layout understanding and subsequent data exploration. The absence of tactile input frequently led to spatial reasoning errors. These findings reveal that tactile templates effectively scaffold mental model construction and enhance spatial reasoning when augmented by LLM assistance.

Abstract

Visualizations are central to communicating data, yet blind and low-vision (BLV) people often lack support for understanding chart types---knowledge that is essential for interpreting new visualizations and collaborating with sighted peers. Prior work found that BLV individuals viewed example tactile charts as more helpful than text-only approaches and preferred them for learning advanced chart types, particularly for understanding spatial layouts and shapes. Meanwhile, large language models (LLMs) are increasingly used by BLV individuals for chart explanation and question answering (QA), but have been studied primarily for dataset exploration rather than chart-type learning. Existing LLM-based chart QA also shows that users frequently ask about layout and structure, yet struggle with spatial concepts and misdirect questions when mental models are weak. We investigate how LLMs influence chart-type learning and whether tactile learning improves subsequent LLM-supported exploration. We extend our tactile chart learning tools with an LLM chatbot that provides interactive explanations and supports follow-up questions. In an interview study with 12 BLV participants, we compare two learning formats: (1) a tactile chart, a textual explanation, and an LLM chatbot; and (2) a textual explanation and an LLM chatbot. The learning phase was followed by exploration of an unfamiliar dataset using alt text and an LLM. Thematic analysis shows that tactile templates support BLV participants' formation of chart-type mental models, which scaffolds subsequent LLM-mediated data exploration. Text+LLM explanations without tactile support show weaknesses for spatial-reasoning tasks.

Problem

• blind and low-vision
• chart-type learning
• tactile charts
• spatial reasoning
• data visualization accessibility

Innovation

• tactile charts
• large language models
• blind and low-vision
• chart-type learning
• mental models