Problem
Research questions and friction points this paper is trying to address.
visual accessibility
low vision
assistive technology
smartphone assistant
AI-based assistance
VisionAssist: An Open-Source Smartphone Assistant for AI-Based Visual Accessibility
🤖 AI Summary
This work addresses the challenges faced by individuals with low vision in independently performing daily tasks due to limited access to visual information. The authors propose and implement an open-source smartphone application that, for the first time, integrates multiple AI-powered visual assistance functionalities—including real-time object localization, image-to-speech description generation, and contextual integration with calendar and contact data—within a unified platform supporting fully voice-driven, hands-free interaction. By synergistically combining computer vision, image recognition, natural language generation, and text-to-speech technologies, the system delivers a comprehensive accessible service. Beyond its practical deployability, the open-source nature of the solution fosters community-driven collaboration, thereby advancing assistive technology innovation and significantly enhancing autonomy for visually impaired users.
📝 Abstract
People with low vision often face challenges in performing everyday tasks that require interpreting visual information. We present \textbf{VisionAssist}, an open-source mobile application designed to improve independence by providing AI-powered visual assistance through a smartphone. The application integrates three complementary functionalities within a single interface. First, it enables users to locate specific objects by analyzing the live camera feed. Second, it generates spoken descriptions of captured images, allowing users to identify visual content such as food labels, documents, and everyday objects. Third, it integrates with the smartphone's contacts and calendar to facilitate emergency calls and provide voice-based reminders. The application supports hands-free interaction through voice commands and delivers all feedback using text-to-speech synthesis, making it fully accessible to users with visual impairments. By combining multiple assistive services into a unified platform and releasing the project as open-source software, the proposed solution aims to encourage community contributions and accelerate the development of accessible technologies. The source code is publicly available at: https://github.com/AOzlemC/LowVisionProject.git
Innovation
Methods, ideas, or system contributions that make the work stand out.
AI-powered visual assistance
open-source mobile application
object localization
image captioning
voice-based interaction
🔎 Similar Papers
No similar papers found.
