Unmanned Aerial Vehicles Control in a Digital Twin: Exploring the Effect of Different Points of View on User Experience in Virtual Reality

2025-10-22

AI Summary

This study addresses safety risks in VR-based teleoperation of drones arising from insufficient situation awareness and poor user experience, specifically investigating the impact of point-of-view (POV) configuration on control performance and subjective experience. In a digital twin–driven immersive VR environment, we implemented real-time teleoperation of a physical DJI Mini 4 Pro drone integrated with multi-perspective VR feedback—namely, first-person, third-person, top-down, and hybrid views—and conducted a systematic comparison using objective metrics (eye-tracking data, trajectory accuracy, stability) alongside validated subjective instruments (SUS, NASA-TLX, IPQ, SSQ). Results indicate that the third-person perspective significantly reduces cognitive load, enhances control stability, and achieves highest user preference; conversely, the first-person view yields smoother motion but exhibits larger trajectory deviation, higher workload, and polarized user acceptance. This work establishes a novel “digital twin + multi-perspective VR” evaluation framework for drone teleoperation, providing empirical evidence and methodological guidance for optimizing VR-enabled human-drone interaction design.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Computer Vision: 3D Computer Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

Controlling Unmanned Aerial Vehicles (UAVs) is a cognitively demanding task, with accidents often arising from insufficient situational awareness, inadequate training, and poor user experiences. Providing more intuitive and immersive visual feedback, particularly through Digital Twin technologies, offers new opportunities to enhance pilot awareness and overall experience quality. In this study, we investigate how different virtual points of view (POVs) influence user experience and performance during UAV piloting in Virtual Reality (VR), utilizing a digital twin that faithfully replicates the real-world flight environment. We developed a VR application that enables participants to control a physical DJI Mini 4 Pro drone while immersed in a digital twin with four distinct camera perspectives: Baseline View (static external), First-Person View, Chase View, and Third-Person View. Nineteen participants completed a series of ring-based obstacle courses from each perspective. In addition to objective flight data, we collected standardized subjective assessments of user experience, presence, workload, cybersickness, and situational awareness. Quantitative analyses revealed that the First-Person View was associated with significantly higher mental demand and effort, greater trajectory deviation, but smoother control inputs compared to the Third-Person and Chase perspectives. Complementing these findings, preference data indicated that the Third-Person View was most consistently favored, whereas the First-Person View elicited polarized reactions.

Problem

• Investigating how different virtual viewpoints affect UAV piloting experience in VR
• Evaluating user performance and mental workload across four camera perspectives
• Assessing digital twin technology's impact on situational awareness and control

Innovation

• Digital Twin replicates real UAV flight environment
• Four distinct camera perspectives tested in VR
• First-Person View increases mental demand but smooths control