Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?

2026-07-23

AI Summary

Existing general-purpose AI-powered visual assistance technologies struggle to meet the complex, personalized, and highly contextual needs of blind and low-vision users. This study investigates how such users can leverage ProgramAT, a natural language–based agent programming tool, to independently create, iteratively refine, and evaluate customized camera-based assistive tools through a two-phase longitudinal co-design process. The work reveals, for the first time, the practical strategies this user group employs to tailor visual assistance via agent programming and identifies key challenges, including mismatches between model capabilities and user requirements. Across the study, five participants developed 37 distinct tools addressing unmet needs—such as identifying Uber vehicles or interpreting hand gestures—and articulated design implications for future systems, including support for conversational scaffolding, community sharing, and integration of specialized models.

Abstract

AI-powered assistive technologies have long supported blind and low vision (BLV) people in everyday tasks, but they are general-purpose and often fall short of meeting complex, individualized, in-situ accessibility needs. Though agentic programming tools, like GitHub Copilot, have the potential to bridge this gap by lowering the technical barriers to building personal AT using natural language, the practical applicability of this creation paradigm has been unknown. We address this knowledge gap through a two-phase longitudinal co-design study with five tech-savvy BLV users using ProgramAT, an agentic programming tool that supports the creation, iteration, and testing of camera-based AT. Overall, co-designers created over 37 tools, with some addressing needs unmet by any existing commercial AT such as identifying Uber rides or interpreting hand gestures. Qualitative feedback from our co-designers and analysis of development logs surface BLV strategies for tool creation, along with key challenges including model capability limits, specification conflicts, and barriers to successful creation. We discuss recommendations to provide appropriate conversational scaffolding, community tool sharing capabilities, and support for specialized models and personal datasets for future agentic programming environments to empower BLV users to create bespoke visual assistance for themselves.

Problem

• assistive technology
• blind and low vision
• agentic programming
• bespoke visual assistance
• accessibility needs

Innovation

• agentic programming
• bespoke assistive technology
• blind and low vision
• co-design
• visual assistance