🔊 Listen to this

The Home As An Operating System.

Today (May 30th, 2017) Andy Rubin, the creator of the Android OS, has unveiled his new company, Essential [1].  Announced today was the Essential phone and the Essential Home.  I will leave it to others to debate the merits of the new phone. I will focus on the Essential Home as it is a step forward in the Voice First revolution.

The Essential Home is a circular design and it should look familiar to any one familiar with Andy’s design language.  Essential Home will be available before the 2017 holiday season and is projected to be $299-$350.  This would make it one of the more costly Voice First devices, however it may match the cost or be a bit less costly from Apple’s Voice First devices that will be released later this year.

The round display is activated in three ways: by a simple voice commands, a tap of a finger or just by looking at it. It offers a number of features that other Voice First systems like Alexa and Google home already provide, however Essential insists its AI is a lot more intelligent.

An example:  playing music, the system can detect when you may have entered into a conversation and automatically lower the volume.  This is a rather important feature that I believe will become a standard in next generation systems.  Additionally, similar to the technology in Nest the system learns your basic daily routines. The AI centered around this is really quite powerful as it creates deeper context on you and how you live.  The system can for example, let you know when something feels off or if a light is left on and even operate as a notification for intrusion.

The timer function on Essential Home, a large use case for first generation Voice First systems, will remind you how long you have until you need to leave and blink the screen with a notification.  There will be deep integrations into your calendar and have maps with live traffic data letting you know you need to leave early for an appointment if traffic is getting backed up.

The system has a deceptively beefy speaker with a fidelity similar to large devices and can connect to additional speakers in an array.  It uses a far-field noise-cancelling microphone array that can do a good job sensing the wake words locally.  The touch screen can be used for control of the device, confirmations and security.  Unique to the device, range sensors can detect when you are looking at it and a number of features can be built around this ability.  Currently, the screen will light up and the system can be put into a post wake word status allowing you to just speak without the need for a wake word.

Essential also has a new interchangeable hardware system, an accessory device bus, where devices that are magnetically connected to the new Phone can also be shared with the Essential Home.  This new system powers the device and also communicates the device with no pairing or software needed to be installed.  This is a rather interesting innovation that can make the utility of the devices that Essential creates.  The accessory device bus specification will also be open source as a specification.

~—~

 

Ambient OS, The First Voice First OS

Essential has built Essential Home on a new operating system called Ambient OS.  The system is designed to be compatible with Alexa, Cortana, Google Assistant and Siri.  The premise is that it will use existing or future APIs to allow for consumers to use the exiting systems if they choose.  This means the system can allow you to reach Amazon though the Alexa API to place an order.  This means in theory Essential Home will have more Voice Apps and Skills than any other single device.

Essential calls this new idea: “the API to your home that enables the creation of applications that extend the reach of a single device”.  This is a light weight OS similar to Apple’s Watch OS but filled with specialized AI and machine learning functionality.  It is a derivative of Android, however it diverges in a number of ways.  Essential Home will have a development environment and an App store.  The developer kit is available today and it is quite unique. For example, the operating system is aware of the layout of your home, a sort of home map is created along with the people in it and the services that your family or roommates are utilizing. This allows it to interact with lighting, speakers and other smart appliances in your home directly across rooms and personal preferences.

Ambient OS extend the reach of a single device and can, for example, allow a timer and have the lights in the livingroom flash when it goes off. The Ambient OS API will give controlled, permission based connection to developers to gain cross platform access to devices, services, and home information and can use these resources as the building blocks of their applications.

Ambient OS will be compatible with 100s of devices at release and will be open source.  And they have already partnered with many companies for the initial release.  The unique aspect of Ambient OS is in the way it sees the entire home as a sort of computer platform with all of the smart systems like lighting and locks as peripherals in the OS.  This is a major leap in a truly new Voice First OS.

There will also be a number of interesting Voice Commerce features that will go a bit further than any device currently in the market.  There will be partnerships with a number of retailers, food establishments and others along with a very unique relationship with a payment company.  The Voice Commerce aspect of Ambient OS will ultimately be a very large feature.

When you are not near or situations are such that you can not talk to Essential Home, there will be a way to send text based command to the system.  There will also be a companion app that can duplicate some of the functions that you can perform locally.

Local AI Hardware

Because Essential Home uses FPGAs to run many AI operations locally, the OS is tuned to do more heavily lifting on the device than and other Voice First system.  Instead of directing all requests via the cloud the Alexa platform and Google Home platform function entirely through the cloud, and gather data from your commands.  Essential Home will limit the data sent to the cloud and it will offer proactive privacy notifications.  Most systems only locally process the “wake word”, e.g. “Alexa, …” the moment a wake word is recognized the phrase you speak immediately after is recorded and sent to the cloud for recognition and intent extraction.   Ambient OS and Essential Home will try to function on most of the recognition locally and seek help from the cloud for some intent extraction and some harder recognition.  Its has the promise of learning to do more locally over time as it gets more context on you.

The local AI aspect of Essential Home may be the largest aspect history will record over the next 10 years.  One of the defining aspects of the Voice First revolution will be who controls your deeper context and where will it live.  Apple has championed this aspect as much as they could with many elements of AI.  However when Siri was released, it was just not possible to do anything but cloud based recognition an extraction.  Amazon’s Alexa system and Google’s Home Assistant do not have facilities to do any local processing other than wake words currently.  As more the Voice First revolution expands, more users will question the guarantees of privacy and Essential as a company is making this a major selling feature.

Andy presented a sort of manifesto on why he is launching Essential and it is quite telling.

“…this weird new world where people are forced to fight with the very technology that was supposed to simplify their lives”—Andy Rubin, 2017

We sense in Andy the frustration of what has become of Android with bloated additions and apps that can not be deleted.  But we also sense the frustration of the actual devices.  Andy is from the school of thought that has defined Apple for the last few decades and as hardware becomes as complex as the OS and Apps that run on it, users have rebelled.  They are downloading less apps (for a lot of other reasons also) and shunning complex hardware, most notably augmented reality and virtual reality goggles.   Here is Andy’s manifesto:

 

Why Essential Home Is Important

The Voice First revolution has started and it can not be denied.  There may have been some cranky skeptics even in January, 2017 but their number has thinned.  Some hold onto the denial of Voice as the next platform in hopes the contrarian stance may play off.  It won’t.  We have entered an age where the device is disappearing and all around us will be ambiance of intelligence.  The screen is not going away, it will be there when you need it and almost like a cast for a broken arm, it will guide you to a level where it will not be needed nearly at all. The keyboard, mouse and screen gestures on glass are not going away, we will just use them less and less.

I wrote a few years back and as recently as 2016 [2] that we will all own a number of Voice First devices and platforms.  They will perform similar and different functions.  Unlike the Mac vs PC and iOS vs Android dichotomies, in the Voice First revolution we will care not of who made the OS, but more about what it can do.  These systems will be in our appliances and our cars and just about everything in-between.  The brilliance of having cross compatibility to any Voice First system that allows API access will serve Essential Home very well.

Andy and the team at Essential see this future and they have built one of the first new Voice First devices and Voice First OSs that welcome it.  Your home, your car and perhaps you will become part of a greater OS environment.  This is a wonderful first step in this recognition and greater direction. Essential will do quite well.

 

____
[1] http://Essential.com

[2] https://medium.com/@brianroemmele/siri-alexa-viv-google-and-more-you-will-use-them-all-cecac5f942ad

~—~