AI Teaches Itself to Multitask in an Ever-Evolving Playground
Artificial intelligence (AI) has already beaten the world’s best human players at chess, Go, and other games. Now, DeepMind is training the systems to play many different games without needing any human interaction data, reveals a new blog by the company.
"We created a vast game environment we call XLand, which includes many multiplayer games within consistent, human-relatable 3D worlds. This environment makes it possible to formulate new learning algorithms, which dynamically control how an agent trains and the games on which it trains," wrote DeepMind.
"The agent’s capabilities improve iteratively as a response to the challenges that arise in training, with the learning process continually refining the training tasks so the agent never stops learning. The result is an agent with the ability to succeed at a wide spectrum of tasks — from simple object-finding problems to complex games like hide and seek and capture the flag, which were not encountered during training."
What does this mean for AI? It means new agents can be created that exhibit behaviors that are widely applicable to many tasks rather than specialized to an individual task, meaning they can adapt rapidly within constantly changing environments. Say goodbye to the issue of a lack of training data and say hello to agents that learn for themselves, redefining reinforcement learning.
How did DeepMind achieve this? They generated dynamic tasks that were neither too hard nor too easy, but just right for training. "We then use population-based training (PBT) to adjust the parameters of the dynamic task generation based on a fitness that aims to improve agents’ general capability. And finally, we chain together multiple training runs so each generation of agents can bootstrap off the previous generation," wrote DeepMind.
The study is called "Open-Ended Learning Leads to Generally Capable Agents" and is available in a preprint version.
Interesting Engineering highlights various Martian geological features, including those discovered by NASA's Curiosity rover over its 10-year voyage.