Thanks to a challenge developed by researchers at Carnegie Mellon University, the entire system of remote control of human-to-human robots has been unlocked. The system is based on reinforcement learning (RL) and enables the coordination of the same people and machines by allowing reconstruction and training from large datasets of human movements.
With just an RGB camera, humans can now operate full-size humanoid robots to perform a variety of tasks, from simple tasks such as picking and placing to efficient operations such as walking, football, and even basketball. This technology bridges the gap between humans and robots, creating new possibilities for empathy and interaction.
Humanoid robots provide a unique remote working ability over time due to their physical similarity to humans. The team’s goal is to convert human movements into human behavior using RGB cameras and record human performance data for robots.
While full-body control of humanoids has been a challenge in robotics, recent developments in reinforcement learning have made significant strides. The team’s Human to Humanoid (H2O) system is a scalable learning-based system that uses only an RGB camera to enable real-time whole-body humanoid robot teleoperation. This breakthrough heralds a new era in teleoperation technology, facilitating intuitive and dynamic interactions between humans and humanoid robots with unparalleled ease and versatility.
“In the future, we can remotely teleoperate a humanoid to perform chores, dangerous missions, and complex tasks that require a human-like level of dexterity and adaptability,” said Tairan He, lead researcher on the H2O project at Carnegie Mellon University. The team’s ultimate objective is to enable the humanoid to mimic as many human-demonstrated movements as possible, opening up exciting avenues for future research in teleoperated humanoid robots.