The next step in operating robots from space

Ready when you are!

The Surface Avatar experiment is a collaboration with the German Space Center (DLR) with three different robots getting ready to go to work at DLR’s German Space Operations Centre (GSOC) in Oberpfaffenhofen, Germany: A lander system with a robotic arm that could load and unload samples, a humanoid robot called Rollin’ Justin, and a robot “dog” called Bert. At ESA’S technical heart ESTEC in The Netherlands, a fourth robot is getting ready, a rover with two robotic arms. They will soon all be together in GSOC, ready for the next step in robotic telepresence, where the robots become an extension of the operator.

Andreas will be controlling the group of robots from the International Space Station during his Huginn Mission, to complete a series of complex tasks.

“The details of the tasks are being kept secret as part of the experiment is to see how Andreas commands the robots with little preparation – an extra challenge for the engineers and the astronaut to make the experiment all the more realistic.” said Marco Sewtz, researcher in Perception and Cognition from DLR.

Scalable Autonomy

Andreas will be the supervisor of the group, giving each robot a task to perform, while he keeps the general overview of the situation on his computer orbiting 400 km above Earth. The Surface Avatar controls include a 7 degrees of freedom joystick with force-feedback to feel resistance pushing against objects and the gravity of objects when Andreas will use the pincers on the rover’s arm. The combination of being the supervisor and moving the robots is called Scalable Autonomy. This way, the astronaut can command the robots how they want to, when they want to.

Operating the robots from the International Space Station comes with a delay as the signal travels from the Space Station to satellites in geostationary orbit, down to Earth and then back to the Station to provide the force-feedback to the controller. This results in a roundtrip delay of 800 milliseconds for operating the robots, which is about four times the blink an eye. On the robots are also cameras that send video back to the Columbus module, allowing Andreas to see how the robot is moving and instruct some of the robots to interact with objects. 

Standing on the shoulders of those before

The Surface Avatar experiment has evolved from several different experiments, merging different approaches to teleoperations and supervised autonomy into one big project.

• Haptics-1, the first haptic experiment from ESA, which was a joystick with one degree of freedom, that was used to determine how to design haptic devices for use in space in the future.
• ESA’s Interact Centaur rover that had two robotic arms and 4x4 wheel drive. Andreas tested this precursor during his iriss mission in 2015, where he placed a round peg in a hole using the Interact rover.
• From DLR, Rollin’ Justin from DLR, the blue humanoid robot with two arms and on wheels that can perform tasks with a large level of autonomy, which former ESA astronaut Paolo Nespoli tested in 2017 and ESA astronaut Alexander Gerst tested as part of his Horizons mission in 2018.
• Bert, the little robot dog from DLR, designed for researching how to animals move and can traverse different terrains, acting as a little scout.

Surface Avatar combines the experience from all of them into one experiment and aims to make the sum greater than its parts, by using supervised autonomy and the force-feedback controls. ESA is setting the groundwork for robotic exploration together with humans. Experiments like these from the Space Station are allowing us to develop the next generation of explorers who could control robots on the Moon and Mars from orbiting outposts above.

Andreas is planned to execute the Surface Avatar experiment in early 2024, while a current astronaut on board the Space Station will conduct it in summer 2023.