This Open-Sourced Low Cost Robot Learns Through Reinforcement Learning

Researchers at Aalto University and OTE Robotics have engineered a low-cost robot that can be used to test reinforcement learning (RL) algorithms, reported Tech Xplore. The bot is called RealAnt and is a real-world model of the 'Ant' robot simulation environment. It is also quite affordable at only 899 Euros (1090 USD).

RELATED: 15 ENGINEERS AND THEIR INVENTIONS THAT DEFINED ROBOTICS

"The initial inspirations for our work were RL studies that successfully demonstrated learning to walk from scratch on ant-like quadruped and humanoid robot simulations," Jussi Sainio, co-founder of Ote Robotics, told Tech Xplore.

"The underlying premise with RL algorithms is that programming a robot to do tasks becomes much easier and more 'natural'—one just needs to define the available sensor measurements, motor actions, then set a target goal and plug them all into a reinforcement learning algorithm, which figures out the rest."

Simulation-based training not required

RL used to require thousands of hours of robot simulation training. However, more recently, researchers have succeeded in teaching robots to walk with very little training data. This means that robots can now be trained in real-world environments without the use of lengthy simulation-based training.

"We quickly realized that RealAnt-like walking robots were not easily and affordably available, especially for reinforcement learning, which can easily damage the robot with abusive controls," Sainio explained.

"There was no complete combined software and hardware stack that one could take and get started with real-world reinforcement learning, compared to the simulator environments. I thus started to build my own robot and interface software prototypes." 

And what impressive prototypes did Sainio build! According to Ote robotics' website, the robot comes with eight Robotis Dynamixel AX-12A smart actuators, a Robotis OpenCM9.04A board, a USB, a tag with reference tag plates, and a 12V 5A power supply.

The website adds that the "RealAnt robot platform from Ote Robotics is designed for real-world reinforcement learning research and development."