VR Multicopter Teleoperation
Third-person VR control of a hexacopter, with onboard SLAM shown to the pilot as a live 3D mesh
Demo video from the SMART Lab at the University of Arizona: the test space, the pilot's third-person view in the headset, and the pilot flying.
First-person-view (FPV) drone piloting suffers from limited spatial awareness - operators can't see the drone's relationship to nearby obstacles. This leads to collisions, especially in confined indoor environments.
The question: can a third-person virtual reality perspective improve obstacle clearance without slowing the operator down?
A live 3D view of the drone and the space around it
In the headset, the pilot sees the hexacopter from a third-person view (TPV), with its surroundings drawn around it as a live 3D mesh. That gives spatial context the first-person view does not.
A ZED camera and the ZED SDK do the SLAM. The pilot wears a Meta Quest 3, and the VR view is a WebXR site built with Three.js, hosted on the hexacopter's onboard Jetson Orin NX 16GB. A Vicon motion capture system provides the flight pose.
My part was the link between the two ends: from WebXR on the Meta Quest 3 to ROS 2 on the Jetson Orin NX, which also runs the ZED camera SLAM.
The VR view is a website. It is built with Three.js on WebXR and hosted on the Jetson Orin NX that flies with the drone. The Meta Quest 3 loads it in its browser.
In first-person view the pilot sees through the drone's camera. In third-person view the pilot looks at the drone from outside, inside the 3D mesh of its surroundings, so the drone and the obstacle are in the same picture.
The hexacopter carries a Jetson Orin NX 16GB. It runs ROS 2 and the ZED camera SLAM, and it hosts the WebXR site that the headset loads.
Software architecture: the ROS 2 back end on the Jetson Orin NX and the Three.js front end in the headset
Key features
By the numbers
In third-person view the pilot kept 0.20 m farther from obstacles: 0.82 m against 0.62 m in first-person view. Task time was comparable, 57 ± 7 s against 55 ± 6 s. Contacts were 0 of 4 in third-person and 1 of 4 in first-person.
That is evidence of technical feasibility, not a usability study: the numbers come from one trained pilot over 8 trials, and the paper itself calls the sample small. A larger study with more participants is the next step.
I'd still choose WebXR for the VR view. It's a Three.js site hosted on the drone's Jetson Orin NX, and the Meta Quest 3 loads it in a browser.
Hughes, J. T., Mazmanyan, G., Ghufran, M., and Rastgoftar, H. "VR-Based Control of Multicopter Operation." IEEE Aerospace and Electronic Systems Magazine, 41(2):46-55, Feb 2026.