XR-Cockpit: A comparison of VR and AR solutions on an interactive training station

One of the most challenging aspects of the implementation of Virtual/Mixed reality training systems is the effective simulation of real-world manipulation of the physical devices included in control interfaces like buttons, sliders, levers, knobs, etc. In this paper we describe a mockup airplane cockpit (XR-Cockpit), featuring interactive components of this kind that demonstrate the feasibility of effective simulations of device manipulation using low cost hand tracking technology and gesture recognition. Based on this system, we performed a user study to compare the effectiveness of the interaction with virtual tools using different visualization solutions: immersive VR, optical and video see-through based MR. In our study, we also checked how well it is possible to perform manipulation of real objects wearing the two video see-through solutions. The analysis of the experimental results provides useful guidelines for the design of Virtual and Mixed Reality training systems involving virtual and physical actions on manipulation devices.