Varjo & Boeing: A New Era in Astronaut Training using Virtual Reality

Connie Miller is a programmer at Boeing Starliner with almost two decades of experience supporting Boeing space programs with new technologies. The Boeing Starliner is developed in partnership with NASA and is designed for researchers and adventurers alike.

Right now, Miller is responsible for progressing the virtual reality training efforts in the Starliner program. Her goal is to set up the team to be able to train astronauts for every step of the mission using immersive technologies.

While NASA’s astronauts have been training for spacewalks in VR for years, the low resolution of existing VR devices has meant that training for the full spectrum of safety-critical scenarios, including operating the spacecraft and docking with the ISS, has not been possible – until now.

When the first crewed mission aboard CST-100 Starliner takes place, the crew will have banked hundreds of training hours for each phase of the entire mission – including launching, docking, re-entering the atmosphere and landing phases – using Varjo’s human-eye resolution VR devices.

The VR headset provides a clear, high resolution system so that the crew can read instruments from a normal distance, which vital to effective astronaut training.

Connie Miller and spaceflight training software engineer Jim May have been exploring the possibility of using VR for astronaut training since 2017, but until last year, the immersive training opportunities were still limited.

But the ambition level was high. Boeing developers in Australia had 3-D modeled the Starliner console using Unreal Engine. In Houston, the virtual training applications were integrated to the physical simulators. Meanwhile, Miller and her colleagues took on the task of testing all available VR devices on the market.

Astronauts need crystal-clear vision to be able to read the display panels in the capsule. The spacecraft’s crew console consists of two displays, each about the size of an iPad, which show mission-critical flight data such as the velocity and trajectory of the aircraft as it moves in space.

For the VR training to be effective, astronauts need to be able to read all the displays simultaneously while operating the simulated aircraft with their hands or controllers. With earlier VR headsets, reading the displays in the immersive training environment was only possible when leaning in close to the displays – but then the astronauts couldn’t see their hands, making it unsuitable for training.

With the human-eye resolution in Varjo headsets, astronauts can see the smallest of details on the crew console. For the Boeing Starliner program, this unlocks unprecedented virtual reality training opportunities for a crewed space mission.

Docking to the International Space Station is one of the most complex operations of a Starliner mission, and is one example of the training scenarios now possible in the immersive VR environment.

Imagine approaching the ISS in the Starliner spacecraft. Your speed is relatively low, but the ISS weighs about 420,000 kilograms. That’s the equivalent of more than 330 automobiles. And even though the ISS is weightless in space, it still has mass. And the spacecraft needs to be steered to a fine point at the docking port by following a cone-shaped path. Projecting the display panels and trajectory data precisely is crucial if VR is to be an effective training tool for such a vital operation.

Varjo’s visual fidelity not only makes it possible to train for precise procedures such as docking, but to practice for unplanned events. VR training allows the Starliner crew to simulate dangerous situations, and build the team’s responses and decision-making abilities, without ever putting the astronauts in danger.

Using VR simulators, the team can perform all the interactions, voice and switch commands, while being immersed in the same surroundings they’d see when sitting in the actual spacecraft.

Ultimately this adds to the safety of the mission.

Augmenting astronaut training with virtual reality has immense operational benefits for Boeing and the Starliner program. Before exploring virtual training, Boeing’s Starliner crew has trained in two state-of-the-art fixed simulators in Houston, with very powerful computers and electronic hardware. Virtual reality allows astronauts to train remotely from anywhere in the world.

With Varjo’s VR devices, astronauts can participate in training sessions remotely, with the same level of realism and interactions as if they were sitting in the physical simulators. The Starliner crew can also perform VR simulations in their individual part-task trainers from anywhere in the world. This brings operational flexibility for the entire team.

Virtual reality also unlocks the ability for astronauts to train while in pre-launch quarantine in crew quarters, which is not possible with more conventional training systems.

With the software development largely behind them, the Boeing team is making its final evaluations and crews are set to begin using the headset full-time in their training.

In the future, the Boeing Starliner team’s ambitions for VR training are likely to reach even greater heights. According to Connie Miller, the team would like to enable flight crews to take the VR training system into orbit aboard the Starliner. This would mean never-before-seen remote training – conducting VR simulations from outer space.

Before this, though, the Boeing Starliner program’s pioneering work in VR simulations is likely to be applied to other safety-critical scenarios.

VR training translates well to other vehicles as well. This platform could potentially be used on other Boeing aircraft.