Frank De Winne's WEAR training session

Life aboard the International Space Station is hard work. Crewmembers have a multiplicity of complex tasks, potentially involving thousands of tools, components and other items. But ESA astronaut Frank De Winne has begun testing the prototype of an unusual helper designed to make astronaut life easier.

The ESA-designed Wearable Augmented Reality (WEAR) is a wearable computer system that incorporates a head-mounted display over one eye to superimpose 3D graphics and data onto its wearer’s field of view.

Controlled by voice for hands-free operation, WEAR includes onboard location and object identification to show astronauts precise information about what they are looking at, as well as providing step-by-step instructions to guide them through difficult, lengthy procedures.

“At the moment, International Space Station (ISS) crews still use paper instruction manuals for many operational and maintenance tasks,” explains Luis Arguello of ESA’s Modelling and Simulation Section, overseeing the WEAR project. “Obviously, it’s easier to perform a task while holding instructions in your hand. So we have developed a new type of user interface that is easier still, allowing astronauts to be guided precisely in their work without holding anything at all.”

From virtual to augmented reality

User Manuals and procedures on board ISS

While the better-known concept of ‘virtual reality’ (VR) concerns immersive artificial worlds, ‘augmented reality’ (AR) involves blending the real world with simulated elements. The technology has already been employed in niche areas such as flight simulations and surgical training, but AR’s range of uses has increased as available computing power has grown.

“The WEAR concept started as research and development within ESA’s General Support Technology Programme, targeting commercial non-space applications such as architecture and maintenance as well as space activities,” explains Mr Arguello. “It’s the result of many years of VR/AR research. The system took a long time to conceive but was very fast to implement: once WEAR was accepted by ESA and NASA, we had to work fast to turn a design concept into flight-ready hardware."

Reflecting the pace of development, WEAR has been assembled from largely off-the-shelf components. Key hardware elements include a mobile computer connected to a headset with a head-mounted display, a pair of video cameras – for barcode reading and objection recognition – and an Inertial Measurement Unit (IMU).

The image-recognition system allows WEAR to know where its user is located, by checking his viewpoint against stored 3D information about the module, derived from a Columbus Computer Aided Design (CAD) model. The IMU works like the orientation tracking technology on an Apple iPhone, letting WEAR assess in which direction its wearer is moving and update his position and the image displayed accordingly.

On the software side, WEAR incorporates a 3D AR toolkit as well as speech recognition and synthesis technology, object recognition and tracking systems and commercial barcode reading technology. Reading the barcode allows the quick identification and retrieval of information of ISS items stored in the onboard Inventory Management System.