This MIT Robot Taught Itself to Play Jenga

The robot combines vision and touch to learn the game in real time.

This MIT Robot Taught Itself to Play Jenga

The evolution of robotics has gone at an incredible pace in the past few years resulting in everything from robots who can hold doors to robots who help humans with the maintenance of other robots. One of the most sought after skills in robotics, however, is still the ability for robots to learn in order to adapt better to their tasks and environments.

Playing Jenga

Now, MIT has introduced a robot that does that with the famous game Jenga. Before you think this might be an exaggeration you may want to read the description of the robot's antics from MIT's announcement.

"In the basement of MIT’s Building 3, a robot is carefully contemplating its next move. It gently pokes at a tower of blocks, looking for the best block to extract without toppling the tower, in a solitary, slow-moving, yet surprisingly agile game of Jenga," says the college's press release.

For the uninitiated, Jenga is a game of physical skill that sees players remove blocks off of a tower of 54 pieces and place them above. The game continues until the tower becomes too unstable and falls.

The robot looks rather simple to be able to play such a complex game. It only features a soft-pronged gripper, a force-sensing wrist cuff, and a camera. And yet what it achieves is nothing short of impressive.

Our Favorite Robots From China's 2018 World Robot Conference

Our Favorite Robots From China's 2018 World Robot Conference

Every time the robot pushes a Jenga block, a computer is fed visual info from the camera and tactical info from the cuff. The computer then analyses this data combined with the moves the robot previously made and deduces the possible outcomes, finding a safe place to place the next block without disturbing the structure. 


“Unlike in more purely cognitive tasks or games such as chess or Go, playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing, and aligning pieces. It requires interactive perception and manipulation, where you have to go and touch the tower to learn how and when to move blocks,” says Alberto Rodriguez, the Walter Henry Gale Career Development Assistant Professor in the Department of Mechanical Engineering at MIT.

“This is very difficult to simulate, so the robot has to learn in the real world, by interacting with the real Jenga tower. The key challenge is to learn from a relatively small number of experiments by exploiting common sense about objects and physics.”

Compared to human players

But how well does the robot play the game? Lucky for us, the researchers were curious about that as well and compared it to actual human players.


“We saw how many blocks a human was able to extract before the tower fell, and the difference was not that much,” says study co-author Miquel Oller.

Before you think this is a fun but rather useless skill, the researchers are keen to point out that it actually has other varied potential applications such as separating recyclable objects from landfill trash and more.

“In a cellphone assembly line, in almost every single step, the feeling of a snap-fit, or a threaded screw, is coming from force and touch rather than vision,” Rodriguez says. “Learning models for those actions is prime real-estate for this kind of technology.”

The study has been published in the journal Science Robotics.

Follow Us on


Stay ahead with the latest science, technology and innovation news, for free:

By subscribing, you agree to our Terms of Use and Privacy Policy. You may unsubscribe at any time.