What is negative training (and how to avoid it in VR)?
Negative training occurs in aviation when training results in the acquisition of knowledge or behaviors that the trainee should not react to or perform in real life. Negative training requires remediation to correct afterward as in the worst case these incorrect behaviors can lead to serious accidents and loss of life.
One practical example of negative training could be as follows: a pilot candidate is training for jet flight in a simulator. If the flight dynamics of the simulated aircraft do not properly represent how a real aircraft would respond to controls in the real world in the same conditions, there is a risk of negative training that will subsequently need to be corrected in live training.
Negative training has been a significant topic in the aviation industry for decades. While simulator training is becoming increasingly sophisticated, the volume of simulator training is also increasing so the need to address potential negative training outcomes is very topical.
What Causes Negative Training?
Negative training can occur for a variety of reasons. It often occurs when the configuration of the simulator does not match the actual aircraft. Old versions of aircraft avionics are often found in simulators where upgraded systems are found in the aircraft. The inconsistencies can contribute to pilot error. Limited simulator fidelity can also cause negative training. Insufficient fidelity in flight models and avionics simulations can result in the acquisition of negative skills and abilities. While simulators can be very good at replicating aircraft operations in standard flight conditions, they are not always as good at simulating some of the more extreme conditions that result in accidents. If trainees and instructors are not presented with these kinds of scenarios, it can lead to learning incorrect behaviors, i.e. negative training.
With the emergence of head-mounted display-based simulators, negative training can be induced due to hardware limitations. Most virtual reality headsets on the market have been made with consumer-oriented use cases in mind (such as gaming and entertainment). The use of these devices can result in negative training since the resolution provided by these devices does not allow pilots to see the critical marking, lights, and objects at realistic visibility ranges. This makes consumer devices less than ideal for demanding simulator training.
Why Not Train Only with Real Aircraft?
Given the issues that negative training can cause, why not just train in real aircraft? After all, most pilots need to train in real aircraft at some point. The main issues are operating costs, availability of aircraft, and ability to train in adverse conditions that are not readily replicable in the real world or are too dangerous to practice in the real world. For example, recovering from a multi-engine failure in a snowstorm is not the most prudent task to practice in the real world since it’s likely that a pilot will need a couple of attempts to get it right.
The biggest issue is that all real aircraft have significant operating costs, and budgets per pilot are limited. It is simply not feasible considering the cost and safety to train pilots in high-operating cost aircraft right from the start. The operating cost of these aircraft is tens of thousands of dollars per hour and it is not safe for inexperienced trainees to operate these aircraft in the types of conditions they need for proper emergency training. Some maneuvers or situations are too risky to train in real-life aircraft so using simulators can fill the gap.
Capacity needs to be taken into consideration. The number of advanced aircraft such as fighter jets or commercial airliners is limited so sufficient training velocity would be hard to achieve without additional training methods.
Finally, not all aircraft are identical or have similar avionics so using different kinds of aircraft might teach a trainee incorrect behaviors for operating another type or configuration of aircraft.
Negative Training in Virtual Reality Based Training Devices
Negative training has been an identified issue in head-mounted display-based training systems. In the past, when pilots trained using VR-based devices, they often became nauseous and were presented with visual content which was not representative of the real world, and they were forced to use equipment and procedures which did not replicate actions done in a typical operation. This resulted in all sorts of negative training which required remediation.
This negative training can include, for example, incorrect use of controllers, gazing in the wrong direction, failing to check specific gauges or instrument panels, and a general aversion to using the training system.
Several factors can cause these behaviors:
- Insufficient headset resolution: Virtual control panels and gauges are unreadable if the display resolution is not sufficient. Distant objects appear as indistinguishable pixelated mush or disappear entirely when they would be easily perceivable in real life. This limits the types of training that can be performed with VR-based devices.
- Excessive visual system latency: Excessive lags in the time when a pilot initiates action to when the result is visible in the headset is one of the major causes of simulator sickness.
- Improperly adjusted inter-pupillary distance: Another major cause of simulator sickness is an improperly adjusted IPD. Many headsets don’t support IPD adjustment or require manual IPD adjustments which are typically ignored by users. This is one of the major causes of simulator sickness.
- Lack of tactile feedback: Development of muscle memory is a critical factor for many training scenarios which is challenging to develop without the ability to interact with physical objects in the exercise. So the development of muscle memory is limited when fully immersed in VR.
- Requirements to use physical controllers to interact with the virtual environment: Rather than using natural movements and gestures, interactions in the virtual environment often require the trainee to use handheld controllers with buttons that differ from what is used in the real aircraft. This will result in incorrect learned behaviors not reflective of real-world aircraft.
- Challenges to trainee assessment: Although the instructor can follow from a monitor what the trainee is seeing, without accurate eye tracking it is hard to determine a trainee’s exact gaze point. While they might generally be looking in the right direction, it can be hard to estimate if a trainee is looking precisely at the correct instrument panel, scanning the horizon, or something similar.
How To Ameliorate Negative Training in VR-based Solutions
Where commodity VR headsets for training are attractive from a cost perspective, the potential for negative training rapidly outweighs the investment needed for an enterprise-class headset. Varjo’s headsets such as XR-3, VR-3, and Aero employ several technologies that efficiently address issues causing negative training in head-mounted displays.
- Human-eye resolution. Varjo headsets address insufficient resolution issues by supporting an industry-leading resolution (over 70 pixels per degree on the XR-3 and VR-3, equivalent to human eye vision). This ensures that objects near and far can be seen accurately, from instrument panels to other simulated aircraft even over a mile away.
- Low-latency video pass-through available for mixed reality. Mixed reality supports the ability to interact with the physical world while fully immersed in a virtual environment. This ensures reactions feel natural and intuitive, and the low latency ensures using the headset causes no nausea. It supports the development of muscle memory and develops accurate reaction times. It enables using real cockpits and instruments as part of the simulator so the trainee can operate the actual controls while flying in a simulated scenario.
- Hand tracking in both virtual and mixed reality scenarios. With hand tracking, trainees can use their own hands and natural gestures without gimmicky VR controllers and can even use a HOTAS system in a very natural way for increased immersion.
- Built-in integrated eye tracking. Varjo headsets support the most accurate eye tracking available in immersive technologies capable of capturing even the most minute eye movements. It allows the trainee’s gaze to be accurately monitored and recorded for review and assessment. It lets the instructor accurately assess if the trainee uses their gaze correctly without needing to resort to guesswork.
- Automatic interpupillary distance adjustment. All Varjo headsets provide an automatic IPD adjustment when a user puts on the headset to ensure visual data is properly presented to the user. This dramatically reduces instances of simulator sickness.