Google Assistant Gets Smarter, Gets A Lens And Does Voice Commerce.


Reading Time: 7 minutes

 

Google Makes A Sweeping Expansion To The Google Assistant Platform.

Today May 17th, 2017 Google announced a spectrum of new Voice First initiatives and products.  Just over a year ago Google presented Google Home and today they have robustly enhanced its abilities.  Google assistant is now on over 100 million devices and used by over 50% on a regular basis.

Here is a highlight of new features:

  • The Google Assistant will accept keyboard input and Voice input on phones.
  • The Google Assistant will have Google Lens camera input that can identify objects and let you ask questions about them.
  • Google Actions — the thing that lets the Assistant interact with third-party services like Alexa skills — are going to be available on phones.
  • You can call any phone number for free in the US and Canada with Google Home.
  • There are 70 different smart home manufacturers that will work with Google Home, with an open developer platform to add more.
  • Google Home can add calendar events and reminders and other proactivity, and will pulse a light when a reminder is waiting for you.
  • Spotify’s free is coming  to Google Home along with Deezer and SoundCloud.
  • Third-party hardware makers will be announcing new versions of a Google Home-like device.
  • You can also tell Google Home to watch HBO or Hulu on Chromecast on your preferred screen
  • Google Home will be able to send visual search results to the Chromecast or other screens when appropriate, e.g.: see your calendar for the day.
  • You can now set reminders and calendar events on Home.
  • Google Home is coming to more countries: Australia, Canada, France, Germany, and Japan.
  • Google Assistant will work in more languages: Brazilian, Portuguese, French, German, and Japanese. Later this year Google will add Italian, Korean, and Spanish.

More Accurate Speech Recognition

The accuracy of the speech recognition technology on Google Assistant has improved significantly, just a 4.9% error rate.  This error rate is actually better than average conversational human recognition in common conversations. Google also announced that 70% of Voice based Google searches are not just keywords one would type, but dialogues one would have in a conversation.  This is a fundamentally important shift in users and use cases.

Google Assistant uses neural beam forming technology that allows for two microphones plus highly optimized AI and Machine Learning that allows for an even higher accuracy.  As the accuracy improves the user experience increase dramatically.  Google has been quietly improving this accuracy as more people use Google assistant.  This AI is also being applied to Google Home and it shows as the recognition that was good when released, has improved substantially over the last few months.

 

Google.AI In The Cloud With Tensor Processing Units 

The technology is enhanced by Tensor Processing Units (TPU), a highly optimized customized system that is designed to vastly accelerate AI and AI and Machine Learning. Running Tensor Flow exclusively allows TPUs to focus on the large dataset required for neural network building. Google is making these modules available to developers via Cloud TPUs.  Google also announced Google.ai and AutoML tools that will allow easy access for developers that do not have AI skills to uses these new tools.  This will unleash an explosion of AI extension deeply into existing software but also as the front end of new and existing software.

Google is using TPUs internally to enhance every aspect of the products the company is producing.  Announced today is Google Lens.  Google Lens will give image recognition to the Google Assistant.  This is a very powerful system that will allow you to simply ask “What am I looking at?”.  Google Lens will reach into the neural network cloud and not only identify what you are seeing but apply context and content related to the results and speak it back to you or present the results on a situational screen connected to your Google Assistant account.

 

Google Gives Voice First Systems A Lens

Adding visual aspects to Voice First systems is going to be a big thing in 2017 and beyond.  This allows for far more use cases where ironically, you do not need to be tied to a screen.  The AI in Google Lens will inform the Voice First AI in Google Assistant and allow for real-time context and information to inform you of useable knowledge.  One could use this for real-time language translation on to Voice Commerce situations where you simply point your camera and say “Buy this”.   We will dive into Voice Commerce in a bit.

Google has also extended Google Assistant to become text ready.  This means that text and Voice can be an input.  There are situations where Voice is not ideal and text input to Voice First systems are a requirement.  This actually may create a bit of a ChatBot use case subsystem over time.  Although I do not see ChatBots becoming a major feature.

 

Google Assistant Developer Platform 

Google is opening the entire Google Assistant API to developers to create version for other devices.  This is similar to what Apple has done with the Alexa platform. The Google Assistant API will expand the Voice First platform to many new devices, appliances, automobiles and other products. Manufactures can get qualified for a “Google Assistant Building-in” logo and registry.

 

Google Home Free Phone Number Calling

Google announced the ability for Google Home devices to create Voice First calling to any US phone number for free.  Following the recent announcement from Amazon, that is limited to device to device or device to app.  Google is using the knowledge and infrastructure from Google Voice to make this a seamless hands-free process.

 

Google Assistant Proactivity, Please Don’t Call Them Notifications

Google Assistant Proactive allows for permission based notification via Google Home and other modalities to receive relevant messages with real-time context and continuity.  Just yesterday Amazon released notice they would start the process of proactivity on the Alexa platform.  To think of this feature as “Push notification for Voice” would be not accurate.  Perhaps early versions will represent this characterization, however over time these systems, if they are deployed correctly will have deep context and become more a dialogue started by the Voice First system than a notification we have become almost immune to on our smartphone screen.  I have built a number of systems that focus on this ability and it has become one of the most valuable aspects of my Voice First designs.

The Rise Of Voice Commerce And The Google Assistant Approach 

One of the most powerful aspect to today’s announcement is the Voice Commerce aspect.  Google is very well aware that the pay-per-click model of advertising that brought about the Web First and Mobile First revolutions will not work with the Voice First revolution.  I spoke to this in a Forbes article called “Voice-First Technology Is About To Kill Advertising As We Know It” [1]:

 

After about ten years of me presenting this inevitability, indeed now Google (and maybe a few other companies) seem to be aware:

“…one thing that we are all clear about is the days of three top text ads followed by ten organic results is a thing of the past in the voice first world“— Sridhar Ramaswamy, Senior Vice president of advertising and commerce, November 29, 2016

During an investor call on November 29, 2016 Sridhar Ramaswamy, Senior Vice President of Advertising and Commerce at Google, spoke to the coming shift of their business model as voice first device interactions begin to dominate our lives. Web-based search will never fully disappear, but a generation of kids are growing up around the Voice First revolution and fully expect a computer to be something they talk to and talks back.

In this highly controversial article, I spoke to the transformation the Voice First revolution will have to advertising.  I have stated for over 3 decades that we will reach a point where commerce will cut around advertising.  I have called this Voice Commerce.  Over the last 5 years I have met with 100s of payment companies and merchants to present this inevitability—many thought me insane.  Today it seems to be less insane and more a massive disruption.  Merchants realize that selling will radically change in these new environments, for example there are no “shopping carts”.  Astute payment companies have been listening to and viewing many of my designs that present how Voice Commerce will take hold.  Today the less astute companies hopefully have fully woken up.

 

~—~

~—~

 

The Amazon 1-click patent and system defined the Web First and Mobile First revolutions.  Clearly it was so powerful, Apple was one of the few early companies to get a license from Amazon to use it.  One could say it one of the fundamental reasons Amazon has been so successful and to some extent Apple and the iTunes/music/movie/app store has been so successful. Not surprisingly the most successful retailer in the world, Amazon has created Echo/Alexa and a Voice Commerce platform.  Although Amazon demurs the Voice Commerce element of the Alexa platform, it will be seen by history as a great “head fake”.

Google’s seamless Voice Commerce system is a just one way merchants can deploy transactions in a Voice First setting. The system relies upon the visual systems built into the chat-like Google Assistant interface.  During today’s demonstration an order at Panera is presented.  The flow is simple and intuitive and completely driven by Voice.

The transaction is currently tied in this demonstration to Google Payments and the indigenous payment system from this merchant.  In the future discovery, ordering and payments will also take other paths.

Food ordering is one of the lowest hanging fruits on the Voice Commerce tree.  I have stated that by 2023, 50% of food ordering will be via Voice First, Voice Commerce systems.  Google is making sure they are a primary player in this new modality.  The Google Lens and TPUs will be able to extend Voice Commerce in many unique ways.  In the future you can just point at a product or location and order the product or initiate a transaction from the merchant location the Google Lens is pointing to.  We will also see similar use cases we find in the Amazon Look for fashion recommendation and other unique cases I have been working on.

The Google developer tools for Voice Commerce are quite powerful.  Google will make their payment gateway available for free to developers.  This is a fundamental shift in how payments may be selected by developers as they build their Voice Commerce apps.  The developer will very likely use the default payment platform if payment companies do not find a way to offer a better experience.  This may seem impossible at first, however I have been working on Voice Commerce payment platforms for a number of years and I can say there are many opportunities for legacy companies and startups.

Google also released robust developer access to Account Linking, re-engagment and order update tools for Voice Commerce apps.

 

The Google Voice First App Store

Google has built an integrated app store that allows for discovery and invocation of new Voice Apps.  Google’s approach is similar to Apple’s approach in that they built it into the chat application of Google Assistant.  Apps are seamless to invoke and we can already see a path for some monetization in the future.

Voice First Systems, The New Electricity Forming A Foundation To Everything

Today we have seen a far more aggressive Google.  Extending the AI First mantra into many products and services.  Google has the platform to become a formidable company in this sector.  However, quite unlike the PC revolution and the Mobile revolution, the Voice First revolution will not be an “either-or” situation for most in the future.  The cost of the Voice First devices will plummet to a very low cost or subsidized to to be free.  They will also inhabit our appliances and other devices including our smartphones.  They will live on an abstraction layer onto of competitor’s products (e.g.: Google Assistant on iPhone).  Along the way Voice First systems will be the new electricity forming a foundation to everything.   Google has been a form of electricity for the web, today they are moving to make it bigger.

 

~—~

~—~

______

[1] Voice-First Technology Is About To Kill Advertising As We Know It

Of interest: