Enhancing Interaction with Augmented Reality through Mid-Air Haptic Feedback: Architecture Design and User Feedback
🤖 AI Summary
This work addresses the spatial misalignment challenge between haptic feedback and virtual objects in augmented reality (AR). We propose the first flexible, cross-platform architecture enabling deep integration of mid-air haptics (Ultrahaptics) with AR, supporting HoloLens, iOS, and diverse haptic devices—including wearable, grasp-based, and ultrasonic mid-air systems. Our approach leverages AR spatial registration, haptic-visual synchronized rendering, and cross-device semantic mapping to achieve high-fidelity haptic representation and spatial consistency of virtual objects. User studies demonstrate that mid-air haptics significantly improves shape recognition accuracy (+32%) and scaling task completion rate (+41%). We validate the architecture’s feasibility through two applications—Form Inspector and Simon Game—and uncover systematic user expectation mismatches regarding haptic metaphors (e.g., virtual buttons), thereby informing the evolution of haptic AR interface design principles.
Application Category
📝 Abstract
Nowadays, Augmented-Reality (AR) head-mounted displays (HMD) deliver a more immersive visualization of virtual contents, but the available means of interaction, mainly based on gesture and/or voice, are yet limited and obviously lack realism and expressivity when compared to traditional physical means. In this sense, the integration of haptics within AR may help to deliver an enriched experience, while facilitating the performance of specific actions, such as repositioning or resizing tasks, that are still dependent on the user’s skills. In this direction, this paper gathers the description of a flexible architecture designed to deploy haptically enabled AR applications both for mobile and wearable visualization devices. The haptic feedback may be generated through a variety of devices (e.g., wearable, graspable, or mid-air ones), and the architecture facilitates handling the specificity of each. For this reason, within the paper, it is discussed how to generate a haptic representation of a 3D digital object depending on the application and the target device. Additionally, the paper includes an analysis of practical, relevant issues that arise when setting up a system to work with specific devices like HMD (e.g., HoloLens) and mid-air haptic devices (e.g., Ultrahaptics), such as the alignment between the real world and the virtual one. The architecture applicability is demonstrated through the implementation of two applications: (a) Form Inspector and (b) Simon Game, built for HoloLens and iOS mobile phones for visualization and for UHK for mid-air haptics delivery. These applications have been used to explore with nine users the efficiency, meaningfulness, and usefulness of mid-air haptics for form perception, object resizing, and push interaction tasks. Results show that, although mobile interaction is preferred when this option is available, haptics turn out to be more meaningful in identifying shapes when compared to what users initially expect and in contributing to the execution of resizing tasks. Moreover, this preliminary user study reveals some design issues when working with haptic AR. For example, users may be expecting a tailored interface metaphor, not necessarily inspired in natural interaction. This has been the case of our proposal of virtual pressable buttons, built mimicking real buttons by using haptics, but differently interpreted by the study participants.
Problem
Research questions and friction points this paper is trying to address.
Haptic Feedback
Augmented Reality
User Interface Design
Innovation
Methods, ideas, or system contributions that make the work stand out.
Haptic Feedback
Augmented Reality
Customizable Design
🔎 Similar Papers
No similar papers found.
