Researchers at the University of California, Berkeley, have developed a human-sized robot using artificial intelligence (AI) techniques to autonomously learn how to walk in the physical world. The teal-colored robot has demonstrated the ability to navigate varied terrains on the university campus, showcasing its potential for operation in diverse environments. However, the robot has not yet mastered the skill of navigating steps.
Humanoid robots are gaining attention for their versatility in various settings, from homes to factories. To enhance their effectiveness, these robots must adapt to changing environments and requirements. The researchers, Ilija Radosavovic and Bike Zhang, explored the application of “reinforcement learning,” a concept popularized by large language models (LLMs), to teach the robot adaptive behaviors. They chose walking as the basic human function to test their theory.
The researchers initiated the learning process in a simulated environment, using Isaac Gym, a high-performance GPU-based physics simulation. Billions of scenarios were run in the simulation, where the algorithm rewarded actions that mimicked human-like walking and penalized those that did not. Once perfected in the simulation, the learned behavior was transferred to the real-world humanoid robot without requiring further fine-tuning.
The core of the learning system is a deep-learning model known as a causal transformer, widely used in LLMs to predict sequences of data elements. Deploying this model allowed the robot to learn from observations and actions, predicting the consequences of its actions in new landscapes it had never encountered before.
Over the past year, the humanoid robot has successfully traversed the UC campus, adapting to patches of grass and roads. Notably, the researchers observed the emergence of unintended traits in the robot’s behavior. For instance, the robot exhibited human-like traits such as swinging its left arm while moving its right foot forward or taking smaller steps to maintain balance on slopes. Impressively, the robot achieved this without relying on sensors to perceive its environment.
While the robot showcased resilience by maintaining balance when subjected to external disturbances like pokes or thrown exercise balls, there are areas for improvement. The robot’s gait is not as refined as a human’s, appearing jerky, especially on harder surfaces like concrete or asphalt. Moreover, the robot lacks awareness of obstacles until it physically encounters them.
The researchers plan to address these limitations in future developments. While the robot may not yet be an ideal home or factory bot, its self-learning capabilities mark a promising start for the development of adaptive humanoid robots.