Google's AI Assistant Can Now Make Extremely Realistic Phone Calls to Book Appointments

Google CEO Sundar Pichai took to the stage at I/O 2018 to entertain a mesmerized audience with Google Assistant’s new capability. The CEO played a recording of the Assistant booking a hairdresser appointment. 

What took the crowd and all the video’s viewers by surprise was how realistic and natural the Assistant sounded. The system even inserted a “mmmm” in between words just like real people do. 

(Skip to 1 hour and 55 minutes for the demo.)

Google Duplex

The new technology, according to Pichai, is called Google Duplex. This Tuesday, Google released a blog on the novel technology by Yaniv Leviathan, Principal Engineer and Yossi Matias, Vice President, Engineering for Google. 

The blog defined Duplex as “a new technology for conducting natural conversations to carry out “real world” tasks over the phone.” Duplex is designed for specific tasks, such as appointment scheduling, that are best served by a human-like conversational experience.

Achieving this was no easy task. The blog explains that human conversations are riddled with complications such as faster speeds that result in less clearly articulated words and complex sentences with a variety of meanings.

"The amazing thing is that Assistant can actually understand the nuances of conversation."

To overcome all these obstacles Duplex needed to respond to interactions on many levels. “Google Duplex’s conversations sound natural thanks to advances in understanding, interacting, timing, and speaking,” stated Leviathan and Mattias. 

Pichai echoed this sentiment during his demonstration on stage. “The amazing thing is that Assistant can actually understand the nuances of conversation,” he said.

Pichai added that the Assistant can react in an intelligent manner even if a conversation veers off course. “We really want it to work in cases, say, if you’re a busy parent in the morning and your kid is sick and you want to call for a doctor’s appointment,” he said. 

System Operations

Duplex’s effectiveness lies in its system’s operations. According to Leviathan and Mattias, the Duplex system “completes the majority of its tasks fully autonomously, without human involvement.” 

This is due to to the fact that the system was developed under “real-time supervised training” similarly to how a human superior would train a disciple. Google used experienced operators as instructors to adjust the behavior of the system in real time.

Once the system had learned to operate at the desired quality level, it was then able to make calls autonomously. The system is also equipped with a “self-monitoring capability” that distinguishes tasks that can not be completely autonomously by Duplex and diverts them to a human operator.  

[see-also]

Leviathan and Mattias concluded their blog by explaining their motivation for developing Duplex. “Allowing people to interact with technology as naturally as they interact with each other has been a long standing promise,” they wrote. 

“Google Duplex takes a step in this direction, making interaction with technology via natural conversation a reality in specific scenarios. We hope that these technology advances will ultimately contribute to a meaningful improvement in people’s experience in day-to-day interactions with computers,” Leviathian and Mattias stated.