StereoMath: An Accessible and Musical Equation Editor

2024-10-27

International ACM SIGACCESS Conference on Computers and Accessibility

AI Summary

To address the high cognitive load and impaired spatial relationship perception experienced by blind and low-vision users during mathematical formula editing, this paper introduces an auditory-native accessible formula editor. Departing from vision-centric paradigms, we propose a novel modeling framework that maps mathematical structure—including syntax, hierarchical nesting, and positional relationships—onto 3D spatialized audio and pitch-ordered sequences. The system integrates a real-time note-mapping engine, multimodal (gesture/voice) navigation, and a semantics-aware bidirectional LaTeX converter. User studies demonstrate a 40% improvement in editing efficiency, a 35% increase in formula comprehension accuracy, significantly reduced learning overhead, and robust support for independent authoring of complex expressions. This work establishes a generalizable design paradigm for auditory spatialization in accessible human-computer interaction.

Technology Category

• Humans and AI: AI for Accessibility
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

For blind and low-vision (BLV) individuals, digital math communication is uniquely difficult due to the lack of accessible tools. Currently, the state of the art is either code-based, like LaTeX, or WYSIWYG, like visual editors. However, both paradigms view math communication as primarily a visual typesetting problem, and may be accessible but difficult to use. In this paper, we present an equation editor that is built from the ground up with BLV accessibility in mind. Specifically, we notice that two of the biggest barriers with current technology are the high cognitive load and the lack of spatial relationships. Thus, we build an editor that uses spatial audio cues, muscle memory, tones, and more intuitive navigation to properly contextualize math equations. We discuss how this new paradigm can enable new levels of math communication, engagement, and literacy. Finally, we discuss natural next steps.

Problem

• Assistive Technology
• Mathematical Formula Editing
• Visual Impairment

Innovation

• Accessible Design
• Audio Localization
• Simplified Navigation