Navigation Framework for Blind and Visually Impaired Persons based on Sensor Fusion

2025-01-27

AI Summary

To address the safety and efficiency challenges in navigation for visually impaired individuals, this work tackles low accuracy and poor robustness in key subtasks—including obstacle detection, localization, recognition, motion planning, and environmental understanding—arising from multi-sensor fusion. We propose an evaluable and scalable integrated navigation framework. Our method introduces three novel algorithmic components: (1) a tightly coupled homogeneous sensor fusion approach based on the Extended Kalman Filter (EKF); (2) a modular, plug-and-play architecture supporting heterogeneous modalities (e.g., RGB-D, IMU, ultrasonic, and LiDAR); and (3) an error-state EKF (ES-EKF) complementary fusion strategy incorporating prior semantic constraints. Experimental results demonstrate significant improvements: average localization error is reduced by 32.7%, and system failure rate decreases by 68%. The framework delivers a theoretically grounded and engineering-practical multimodal fusion solution for intelligent navigation aids for the blind.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Individuals who are differently-able in vision cannot proceed with their day-to-day activities as smoothly as other people do. Especially independent walking is a hard target to achieve with their visual impairment. Assistive electronic travel aids equipped with different types of sensors are designed for visually impaired persons to assist their safe navigation. The amount of research on combining multiple sensors in assistive navigation aids for visually impaired navigation is limited. Most work is targeted at sensor integration but not at sensor fusion. This paper aims to address how sensor fusion and integration will be used to improve the sub-processes of visually impaired navigation and the way to evaluate the sensor fusion-based approach for visually impaired navigation which consists of several contributions to field sensor fusion in visually impaired navigation such as a novel homogeneous sensor fusion algorithm based on extended Kalman filter, a novel heterogeneous sensor integration approach, and a complementary sensor fusion algorithm based on error state extended Kaman filter. Overall this research presents a novel navigational framework to integrate obstacle detection, obstacle recognition, localization, motion planning, and current context awareness with sensor fusion.

Problem

• Sensor Optimization
• Visual Impairment Navigation
• Obstacle Detection

Innovation

• sensor fusion
• obstacle detection
• navigation assistance for visually impaired