FleetScape: A Mixed Reality Sandtable for Spatial Supervision and Control of Scalable Drone Fleets

2026-07-28

AI Summary

Current drone swarm control interfaces rely on single-vehicle interaction paradigms, which struggle to meet the complex supervisory demands of large-scale formations across spatial, temporal, and safety dimensions. This work reframes swarm control as a spatial interaction task and introduces FleetScape, a novel mixed reality (MR) sandbox system that externalizes multidimensional data—such as mission status, environmental context, and safety constraints—into spatialized visualizations. FleetScape enables seamless transitions between manual intervention and autonomous supervision, integrating high-fidelity building inspection simulation, synchronized multi-drone–environment data streams, and spatial interaction mechanisms. User studies demonstrate that FleetScape significantly enhances operators’ situational awareness and clarifies transitions between control modes; however, as swarm size increases, situational awareness degrades, prompting users to adapt their supervisory strategies.

Abstract

As autonomous drone deployments scale from individual units to coordinated swarms, the human operator's role shifts from direct piloting to high-level supervision. Current interfaces often treat multi-drone control as a scaled-up version of single-drone operation. We instead investigate how reframing fleet supervision as spatial interaction can better support the spatial, temporal, and safety demands of complex missions. We present FleetScape, a Mixed Reality (MR) sandtable system that externalizes layered real-time mission, safety, and environmental data while enabling fluid transitions between manual intervention and autonomous supervision. We developed a high-fidelity building inspection simulation that generates and streams synchronized multi-drone and environmental data for MR visualizations. We used this prototype to conduct a user study with six experienced drone pilots managing fleets of up to 15 drones. Our findings show that FleetScape supports situational awareness through layered spatial representations and clarifies control mode transitions. However, a limit to situational awareness was observed as fleet size increases, leading to different supervisory strategies. Finally, we derive design implications for supporting scalable drone fleet supervision.

Problem

• drone fleets
• spatial supervision
• human-drone interaction
• scalable autonomy
• mixed reality

Innovation

• Mixed Reality
• Drone Fleet Supervision
• Spatial Interaction
• Situational Awareness
• Scalable Autonomy