Researchers Design Machine that Mimics Language Learning in Kids

Deep learning systems are revolutionizing our approach to understanding and mimicking a vast array of processes, some of whose applications are as diverse as video games or even mapping out obesity rates in select cities. It's also proven instrumental in helping to clarify certain processes which are difficult to gain a full perspective on.

Now, a team of researchers from MIT's Center for Brains, Minds, and Machines (CBMM) and Computer Science and Artificial Intelligence Laboratory (CSAIL) have designed a study that seeks to answer some of these questions centered around language learning in young children. Enhancing the process known as a semantic parsing--which consists of converting language into a logical and measurable data form--it essentially employs deep learning algorithms to copy this process in children, achieving results only through observation.

The team behind the research will present the details in a paper at this year's Empirical Methods in Natural Language Processing conference in Brussels, Belgium, which runs between November 2nd and 4th.

Enhancing the Language Process

To achieve the results, the team used video for the training, as they thought it would offer more accurate results. “There are temporal components — objects interacting with each other and with people — and high-level properties you wouldn’t see in a still image or just in language, explains Candace Ross, a graduate student in the Department of Electrical Engineering and Computer Science and CSAIL and first author on the paper. 

In total, roughly 400 videos demonstrating a number of tasks were used, with 1,200 captions being added, thanks to contributions sent via the crowdsourcing platform Mechanical Turk. The scientists then made the wise choice to divide the captions into two groups:

840 would be used for tuning and training purposes, while the remaining 360 were reserved only for testing, offering a streamlined process in which “you don’t need nearly as much data — although if you had [the data], you could scale up to huge data sets,” says co-author Andrei Barbu, a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Center for Brains, Minds, and Machines (CBMM) within MIT’s McGovern Institute.

 Unlocking Clues About Learning 

The promising research offers the possibility to deepen understanding about some of the fundamental learning processes which children engage in. With the obvious challenges that children have with articulating some of these nuances due to their different stage of developmental, AI is playing a valuable role.

“A child has access to redundant, complementary information from different modalities, including hearing parents and siblings talk about the world, as well as tactile information and visual information, [which help him or her] to understand the world,” shares co-author Boris Katz, a principal research scientist and also head of the InfoLab Group at CSAIL.

[see-also]

"It’s an amazing puzzle, to process all this simultaneous sensory input. This work is part of a bigger piece to understand how this kind of learning happens in the world.”

As the process of acquiring language is so complex, it requires a multi-disciplinarian approach that takes into account the world that children inhabit. “Children interact with the environment as they’re learning. Our idea is to have a model that would also use perception to learn,” Ross adds.

The researchers have also shared the details of their paper, titled "Deep sequential models for sampling-based planning", via the Computer Science Department of the University of Washington.