An AI That Could Create Video Games by Just Watching The Gameplays

Artificial intelligence (AI) systems continue to amaze us with their applications based on efficient methods of deep learning, which have resulted in breakthrough approaches in the areas of understanding brain function, and even autism therapy. Moreover, there are many efforts being put into producing literature that tackles some of the general public's concerns and lack of information about what AI can do.

Now, a group of Georgia Tech researchers has designed an AI system that can recreate classic 2D video games simply by studying the way that they are played. And gamers, of course, couldn't be more excited. They've pulled off their project with Super Mario Bros. and Mega Man. Below are images of the original and the copy (the newer version appears without the score and time info in the upper part of the screen).

Developing the Method

To achieve these results, the team used a kind of engine-cloning method which essentially involved only a parser, as well as the machine's efforts to study and memorize the pixels on the screen. In addition, it is given a generous amount of info about (1) a set of concepts that explain the relationship between certain aspects of the game, and (2) a solid group of images compiled from the game which range from the heroes to the villains.

The most astounding part of the results: the system was able to gather enough information in under two minutes. And though the new version isn't a perfect match, it's an impressive copy. 

“For each frame of the video, we have a parser which goes through and collects the facts. What animation state Mario is in, for example, or what velocities things are moving at,” Matthew Guzdial, lead researcher and Ph.D. student in computer science, shares with The Verge. “So imagine the case where Mario is just above a Goomba in one frame, and then the next frame the Goomba Is gone. From that it comes up with the rule that when Mario is just above the Goomba and his velocity is negative, the Goomba disappears.”

A Novel Approach

What sets this research apart from other similar work, serves as a source of pride for the researchers, is that they achieved their results using a method that didn't rely on the code. "Our AI creates the predictive model without ever accessing the game's code, and makes significantly more accurate future event predictions than those of convolutional neural networks," explains Guzdial.

"A single video won't produce a perfect clone of the game engine, but by training the AI on just a few additional videos you get something that's pretty close."

[see-also]

The machine learning system is changing our concept of what the potential is for AI in the area of gaming, although the team acknowledges that more work would need to be done to enhance the capabilities of the system to be used with 3D platformers. 

"The technique relies on a relatively simple search algorithm that searches through possible sets of rules that can best predict a set of frame transitions," says Mark Riedl, associate professor of Interactive Computing and co-investigator on the project. "To our knowledge, this represents the first AI technique to learn a game engine and simulate a game world with gameplay footage."

Details about the innovative study appear in a paper, titled "Game Engine Learning from Video", which the team has made available on the Georgia Tech website.