Reinforcement Learning Framework Enables Robust Walking in Bipedal Robots
Researchers develop a model-free reinforcement learning method that trains robust locomotion policies in simulation and successfully transfers them to the real Cassie bipedal robot.
Researchers present a new model-free reinforcement learning framework to solve the difficult problem of robust bipedal robot walking. Traditional model-based controllers rely heavily on simplifying assumptions and precise modeling, meaning even tiny errors quickly lead to unstable movement. This new approach eliminates those dependencies by training the control policies entirely through simulated trial and error.
To ensure the simulated behaviors work effectively in the real world, the team utilizes domain randomization during the training process. This technique forces the AI to learn strategies that remain stable across a wide variety of system dynamics and unexpected physical variations. As a result, the system successfully bridges the sim-to-real gap without relying on residual control methods used by previous learning approaches.
The trained policies deploy directly onto the physical Cassie bipedal robot and demonstrate impressive versatility and resilience. Cassie successfully executes dynamic walking behaviors that include tracking specific target velocities, adjusting walking height, and performing turning maneuvers. These learned capabilities ultimately make the robot significantly more robust than both traditional controllers and earlier machine learning methods.