Robots Learn Different Locomotion Behaviors Using Human Demonstrations

It's all in simulation at the moment, but plans are to test their method on real robots now.
Fabienne Lang
Humanoid robot locomotionYang et al./University of Edinburgh

Robots are a major part of our future, and researchers around the world have been working hard at enabling smooth locomotion styles in humanoid and legged robots alike. 

Now a team of researchers from the University of Edinburgh in Scotland has put together a framework for training humanoid robots to walk just like us, humans, by using human demonstrations. 

Their study was pre-published on arXiv, and combines imitation learning and deep reinforcement learning techniques. Their findings were also published in IEEE Xplore.


Useful human knowledge into robot systems

"The key question we set out to investigate was how to incorporate (1) useful human knowledge in robot locomotion and (2) human motion capture data for imitation into deep reinforcement learning paradigm to advance the autonomous capabilities of legged robots more efficiently," Chuanyu Yang, one of the researchers part of the study, told TechXplore.

"We proposed two methods of introducing human prior knowledge into a DRL framework."

The team's framework works off of a unique reward design that utilizes motion caption data of humans walking as part of the training process. It then combines this with two specialized hierarchical neural architectures: a phased-function neural network (PFNN) and a mode adaptive neural network (MANN).

"The key to replicating human-like locomotion styles is to introduce human walking data as an expert demonstration for the learning agent to imitate," Yang explained. "Reward design is an important aspect of reinforcement learning, as it governs the behavior of the agent."

You can't help but think of the similarities of training a dog to carry out tricks and rewarding it with a bone afterwards...

Most Popular
Robots Learn Different Locomotion Behaviors Using Human Demonstrations
Teaching the humanoid robot to move its right foot just like a human, Source: Yang et al./University of Edinburgh

The wonderful news about the team's framework was that it even enabled the humanoid robots to operate on uneven ground or external pushes. 

The team's findings suggest that expert demonstrations, such as humans walking, can majorly enhance deep reinforcement learning techniques for training robots on a number of different locomotion styles. Ultimately, these robots could move just as swiftly and easily as humans, also while achieving more natural and human-like behaviors

At the moment all the research has been carried out through a simulation, the next steps involve trying the framework out in real life. 

"In our future work, we also plan to extend the learning framework to imitate a more diverse and complex set of human motions, such as general motor skills across locomotion, manipulation, and grasping," Yang said.

message circleSHOW COMMENT (1)chevron