Google Shows Off Super-Fast 'Next Generation' Voice Assistant

Google says that breakthroughs in how its voice assistant processes language have enabled faster, smarter voice interactions—even offline.

At the Google I/O developer conference, the search giant announced major improvements to its voice assistant that deliver faster and more dynamic results.

The Next Generation

The Google Assistant has been answering questions and turning on smart lights for several years now, but it has always heavily relied on backend support from Google. If you lose your internet connection, the Assistant is basically useless. That's changing with what Google is calling its Next Generation Assistant.

Thanks to breakthroughs made in how Google processes speech, it can now understand what you're saying using on-device resources. That means the Assistant won't need to send information back to Google on the cloud and get results. On stage at I/O, Scott Huffman, VP of engineering for Google Assistant, demonstrated how the Assistant could handle a series of rapid-fire requests to open apps, input information, and even toggle a phone's flashlight on and off.

The Next Generation Assistant can also coordinate actions between apps. On stage, we saw a woman responding with her voice to a text message. The sender asked about a trip she'd taken. The woman opened her photos, asked to see pictures from Yellowstone, reduced the results to only those of animals, and then sent the result via text message, all with her voice.

Part of the power of the Next Generation Assistant is its ability to tell the difference between commands and dictation for text. On stage, the assistant was told to change the subject line in the middle of dictation, and smartly excluded that command from the email body text while carrying it out.

The most critical feature showed on stage was ending an alarm simply by saying "stop" instead of having to preface it with the "Okay Google" trigger phrase. I can confirm that this will be enormously popular in my home.

"The realization of this vision is not far off," said Huffman.

These Next Generation Assistant features will be coming to new Pixel phones later this year.

Safely Offline

Importantly, Huffman says that basic tasks like opening Google Calendar or switching the flashlight on now work without an internet connection at all. This has also been expanded to Live Transcribe, which adds closed captioning to videos in real time and is part of a series of accessibility projects Google is working on.

Stephanie Cuthbertson, a senior director for Android, explained that this used to take a 2GB machine learning model, but the same thing can now be accomplished on-device via an 80MB recurrent neural net.

Running more machine learning and AI activities on individual devices not only means faster results, but more private ones as well. Instead of sending recordings back for analysis, they stay on your phone. Apple boasts of its on-device approach as a major, more private differentiator between Apple and Google—a selling point that may not be so unique anymore.

Smarter, Most Customized, and Everywhere

The Google Assistant will soon have a place in your car. First by coming to Waze and then with an Assistant Driving Mode, which places important information for driving up front in a more accessible format.

Driving Mode will take advantage of the increasingly customized results in the Google Assistant. By sharing the address of his mother with the Assistant, Huffman demonstrated how search results could parse the difference between a "mom's house" being his mother's house and "mom's house" being one of many businesses that share that name. This information, Huffman assured the crowd, can be edited and removed from the assistant from the You tab.