Google quietly hosted a developer show in Europe this week, and one of the best things I’ve seen so far is an exciting update about the future of Google Assistant.

That developer show, called Google Developer Days Europe, was held in Poland on Tuesday and Wednesday, and you can find some videos from the keynotes and sessions on YouTube. Among them is the Day 2 Keynote, in which the firm discusses some of its plans for the improving Google Assistant across devices. (I was tipped off to this information by Android Police.) It’s worth watching.

“Google Assistant is one of the core implementations of an AI-first world,” Google senior engineering director Behshad Bezhadi said during his talk. “We feel that we [Bezhadi leads a team of over 100 engineers building Google Assistant] are part of this next revolution.”

After a short discussion of why AI is such a revolution, Bezhadi argues that AI, in some ways, is making smartphones truly smart, by adding conversational, situational, and intra-device capabilities.

“Conversation is really the core of what we think about when we think about Google Assistant because we think that conversation is the most-used interface human beings have used,” he says. “If we solve the problem of conversational understanding, we can expand it to many use cases.”

Bezhadi then showed some demos of where Google Assistant is at today, using a Pixel smartphone. None of this will be particularly surprising to anyone who has interacted with Google Assistant or any other digital personal assistant: You can have short, meaningless conversations, find out about the weather, discover local attractions, and so on. Google Assistant can also be used to learn about much more specific things: He asked about rides at a particular amusement park, and then asked about the height restrictions at a particular ride at that park.

More intriguing, of course, are the Google Assistant features he demoed that are not yet live. Some of these will launch in the next few months, he said, while others will arrive next year.

“We’re trying to understand more complex natural language,” he said, launching into the following question: “What is the name of the movie where Tom Cruise acts in it and he plays pool, and while he plays pool he dances?” Google Assistant responds, correctly, with some information about the movie “The Color of Money.”

That’s impressive, but Bezhadi also shows how Google Assistant can be enhanced by connecting to other services, either from Google or from third parties.

“Be my Vietnamese translator,” he says to Google Assistant, which says it understands and will translate all of his following sentences to Vietnamese. All he has to do to stop is say, “Stop translation.” This, too, is an impressive demo, and it’s not hard to imagine how this functionality would transform international travel.

Bezhadi goes on to control Google Maps’ Street View (“I would like to be on the top of Eiffel Tower now, can you please bring me there?” as well as a third-party service—“Talk to WebMD”—where he asks about the side-effects of aspirin. He asks implicit questions—“How is my team doing?”—and Assistant uses context about his profile, preferences, and query history (and presumably the time of the year) to answer about the right team.

That latter capability is particularly impressive when you consider that you can teach Assistant about yourself and it will get smarter as you do. This capability is already present in a basic form now, but it will get more capable in an update coming next year by adding natural language learning.

“When the weather is more than 25 degrees [Celsius], I can swim in the lake of Zurich,” he tells Google Assistant. Then, he asks whether he can swim in the lake of Zurich this weekend. “No, you can’t,” Google Assistant replies. “The temperature is less than 25 degrees.”

On a related note, the way that Google Assistant handles context is also going to get more sophisticated. Consider contextual voice recognition: He asks about a particular mountain range. And then he asks about the height of one of the mountains in that range by name. Because the mountain range discussion had happened, Google Assistant understood the context of the next question.

“Contextual voice recognition is going to play a big role,” he said. “Because people always try to continue talking about things which are related, location specific, or contextually relevant.”

In another example, Bezhadi asks about “pictures of Thomas,” but because there is no context, it shows him pictures of Thomas the Tank Engine. So he asks about a sports team roster, and is shown a list of players. So he then asks about Thomas again, and this time he is shown pictures of a player on that team who is named Thomas.

Context is also important for having a continuing conversation. He asks about where the Empire State Building is, and gets an answer. Then asks, “I want to see pictures.” And he is shown pictures of that building. And he asks “how tall is it?” he is given the answer for the Empire State Building. He goes on to ask other questions: “Who built it?” “When?” and “What are the Italian restaurants around there?” and “Call the first one.” It’s a conversation.

Context also works visually (or soon will): Through the integration of Google Lens for visual input, Google Assistant will be able to perform some impressive feats. For example, he points Lens at an apple and then asks, “How many calories?” and Google Assistant tells him how many calories are in an apple. He also did a demo with money in which Assistant converted it to a different currency.

Google is also working on improving voice recognition in noisy environments. With music blaring and the audience roaring, Bezhadi asked questions through his phone, and the Assistant answered correctly. (This one is obviously hard to describe.)

Overall, this was an impressive set of demos. And anyone who bets against Google in this area isn’t seeing the big picture. There’s no question whether Google Assistant wins this war. The only question is which assistant ends up in second place.

Tagged with Google Assistant