April 2022: If man's best friend is a dog, then in the future astronauts' closest companions might well be rovers. A technique allowing astronauts in orbit to control rovers exploring planetary surfaces has been developed by a research team from ESA, the German Aerospace Center DLR and European academia and industry, culminating in an Earth-based rover session commanded from the International Space Station. A paper published in the prestigious Science Robotics journal this week details their results. 1)2)
"This is the first time that an astronaut in space managed to control a robotic system on the ground in such an immersive, intuitive manner," comments Aaron Pereira of DLR.
"Our 6 degree of freedom control interface incorporates force feedback so that the astronaut can experience just what the rover feels, even down to the weight and cohesion of the rocks it touches. What this does is help compensate for any limitations of bandwidth, poor lighting or signal delay to give a real sense of immersion – meaning the astronaut feels as though they are there at the scene."
Let the robots take the strain
Robotics engineer Thomas Krueger, heading ESA's Human Robot Interaction Laboratory, adds: "Robots can be given limited autonomy in known, structured environments, but for systems carrying out exploratory tasks such as sample collection in unknown, unstructured environments some kind of ‘human-in-the-loop' oversight becomes essential. But direct control has not been feasible due to the inherent problem of signal delay – with transmission times constrained by the speed of light.
"So we have been working towards the concept of humans staying safely and comfortably in orbit around the Moon, Mars or other planetary bodies, but being close enough for direct oversight of rovers on the surface – combining the human strengths of flexibility and improvisation with a robust, dexterous robot on the spot to carry out their commands precisely."
"In the end we needed to perform feasibility experiments from space because past research shows that weightlessness can degrade human performance during force and motion tasks," adds Thomas. "This and other unique environmental factors meant that Earth-based simulations would not be sufficient."
Their efforts culminated in the first part of the Analog-1 experiment in late 2019. Astronaut Luca Parmitano aboard the ISS operated the gripper-equipped ESA Interact rover in a mock lunar environment inside a hangar in Valkenburg, the Netherlands to survey rocks and collect samples. The two-hour space-to-ground test was a success, overcoming a two-way signal delay averaging more than 0.8 seconds and a data packet loss rate of 1% plus.
Solving the time factor
"Even though the ISS is in orbit just 400 km overhead, its signals are relayed to Earth via geostationary telecom satellites, then to Europe from Texas via a transatlantic cable," explains Aaron.
"Our team at DLR had to design a control algorithm that could function on a stable basis despite this time lag. Because there is a delay in the force feedback received by the operator, they might continue to move the robot further even after it has hit a rock. This could lead to the robot going out of sync with its controller, potentially vibrating like crazy, perhaps even damaging itself.
"To prevent that happening, we use a concept called ‘passivity' – we look at the overall amount of energy an operator puts in, and on the remote side make sure the robot never gives out any more energy than that, and vice-versa. Like when you push a child on a swing, they are never going to go any higher than the first swing – and with friction and so on will swing gradually lower and lower.
"So for instance, when the robot arm is moving and suddenly hits a rock it would take extra energy to move which the astronaut did not command, so we reduce the command energy at once to slow down the arm. Then, after the 850 microsecond delay, when the astronaut feels the rock they can then choose to add the extra energy to push it.
"This ‘Time Domain Passivity Approach for High Delay' technique is very intuitive in practice and should be able to work well with higher delay times as well."
Figure 6: Rock sampling from space – Analog-1. Analog-1 has been the latest in a series of progressively more challenging human-robot test campaigns involving the ISS, collectively called Meteron (Multi-purpose End-to-End Robotic Operation Network). The first 1 degree of freedom force-feedback test took place back in 2015 with ESA's Haptics-1 experiment, progressing to DLR's 2 degrees of freedom Kontur-2 the following year – advancing now to a full 6 degrees of freedom movement (video credit: ESA)
Next steps on Mount Etna
Thomas concludes: "The main limitation of the work done so far is that our indoor analog lunar environment lacks realism. So this summer will see the second part of Analog-1 take place on the volcanic slopes of Mount Etna in Italy as part of a larger international robotic test campaign called ARCHES.
"Luca Parmitano will once again be controlling the Interact rover, this time in 1G from the ground."
2) Michael Panzirsch ,Aaron Pereira,Harsimran Singh, Bernhard Weber, Edmundo Ferreira, Andrei Gherghescu,Lukas Hann,Emiel den Exter,Frank van der Hulst, Levin Gerdes, Leonardo Cencetti, Kjetil Wormnes, Jessica Grenouilleau, William Carey, Ribin Balachandran, Thomas Hulin,Christian Ott, Daniel Leidner, Alin Albu-Schäffer, Neal Y. Lii, and Thomas Krüger "Exploring planet geology through force-feedback telemanipulation from orbit," Science Robotics, Volume 7, Issue 65, Published: 20 April 2022, https://www.science.org/doi/10.1126/scirobotics.abl6307
The information compiled and edited in this article was provided byHerbert J. Kramer from his documentation of: "Observation of the Earth and Its Environment: Survey of Missions and Sensors" (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (firstname.lastname@example.org).