New Apple Patent. Siri: “Jim the plumber is at your front door. What should I do?


This New Apple Patent Talks About A Voice First Home Hub And More.

On May 11th, 2017 the USTPO released an Apple patent application that is stunning.  This is a landmark patent that describes new embodiments that Apple has never before presented.  I just produced a Multiplex Memorandum for paid subscribers (just $5 bucks a month, you can afford it) on the notable aspects of this patent.  I will present two astonishing new futures.

In this patent Apple makes it clear that the user will be interacting with Siri over a number of devices.  One technique they use is called a blackboard system. A blackboard system is an artificial intelligence approach based on the blackboard architectural model, where a common knowledge base, the “blackboard”, is iteratively updated by a diverse group of specialist knowledge sources, starting with a problem specification and ending with a solution. Each knowledge source updates the blackboard with a partial solution when its internal constraints match the blackboard state. In this way, the specialists work together to solve the problem. The blackboard model was originally designed as a way to handle complex, ill-defined problems, where the solution is the sum of its parts.

The embodiments of this new patent goes into deep detail of how Apple will use the blackboard technology for a new version of Siri.  The patent describes two or more Voice First systems interacting with each other.  In some embodiments the Voice First devices can owned by other users.  This can allow for mediation and orchestration of a number of situations from calendars to solving delegation tasks.

Apple describes one embodiment that is stunning:

[0296] Returning to the first example, Pierre’s digital assistant 700a generated a plan to let Tomas into his apartment. During execution of that plan, the digital assistant 700b associated with Pierre’s apartment awaits contingent input associated with Tomas’ arrival. Referring to FIG. 8Z, as one example of an exception, the digital assistant 700b recognizes that someone who is not Tomas has arrived at the door. The digital assistant 700b requests the name of the visitor, who replies “Jim the plumber.” The digital assistant 700b is expecting Tomas at approximately 8:00 p.m. However, a visitor other than Tomas has arrived. This is an exception to the plan. Referring also to FIG. 7D, according to some embodiments, the digital assistant 700b associated with Pierre’s apartment generates a message that a person who has identified himself as “Jim the plumber” is at the door; that message travels through the blackboard 772 and message gateway 770 to Pierre’s digital assistant 700a, which in this example is shown as digital assistant 700a. Pierre’s digital assistant 700a recognizes the identity discrepancy between Tomas and “Jim the Plumber,” identifying the exception. Optionally, Pierre’s digital assistant 700a first determines if a rule is present in the plan template 802 for handling the arrival of a different visitor; in this example, there is not. Referring to FIG. 8Z, finding no rule, Pierre’s digital assistant 700a handles the exception by causing a message 850 to be displayed on the display 212 of Pierre’s electronic device 200: “`Jim the plumber` is at your front door. What should I do?” Pierre is not expecting a plumber, and is surprised one would want entry to his apartment at 8:00 p.m. Referring to FIG. 8AA, Pierre tells the digital assistant 700a, “Don’t let him in. Also, call the building manager”; the digital assistant 700a displays that message 852 on the electronic device 200 according to some embodiments. Pierre’s digital assistant 700a transmits this message to the message gateway 770, which forwards the message 830 to the blackboard 772. The digital assistant 700b associated with Pierre’s apartment is forwarded the message 830 by the blackboard 772, or retrieves the message 830 from the blackboard 772. The digital assistant 700b then leaves the door locked. Further, the digital assistant 700b calls the building manager, and informs the building manager that “Jim the plumber” is at the door. The plan generated to let Tomas in when he arrives is not complete, because Tomas has not yet arrived. However, the exception has been handled by refusing entry to the apartment to “Jim.”

You read this correctly, in this embodiment Siri is answering the front door and mediating interactions with guests.  This is a far more sophisticated Siri that has high context and using a shared blackboard system to interact.  We can see Apple diving far deeper into the home with ambient Voice First technology.  There is a lot of embodiments that go even deeper into how the new Siri platform will extend in this patent.

In this patent Apple for the first time describes a stand-alone Voice First device they are calling a “human automation system hub”:

[0258] Referring to FIG. 7D, a block diagram architecture 766 is shown for virtual assistant interactions according to various examples. A first digital assistant 700a is shown, substantially as described above with regard to the digital assistant 700. The first digital assistant 700a is accessed by the user through a portable electronic device 200, according to some embodiments. A second digital assistant 700b is shown, substantially as described above with regard to the digital assistant 700. The second digital assistant 700b is accessed by a user through a human automation system hub, according to some embodiments. According to other embodiments, the second digital assistant 700b is accessed by a user through a portable electronic device 200, a television interface such as the Apple TV.RTM. digital media extender of Apple Inc. of Cupertino, Calif., a vehicle, or any other device through which a digital assistant 700 can be accessed.

The user digital assistant #700a is separate from the hub digital assistant #700b; the two digital assistants 700a, 700b are discrete entities.

Each separate digital assistant 700a, 700b is associated with the same user, according to some embodiments; for example, the user digital assistant 700a may be accessed by the user through his or her portable electronic device 700a, and the hub digital assistant 700b may be accessed by the user through his or her home automation system.

It is very clear Apple is directly speaking about the continuity between a number of access points to this new Siri environment.

This new patent offers some of the most interesting aspects of how Apple will move the new Siri forward.  I called this a landmark patent for many reasons.

Patents offer an aperture into the future.  I have studied Apple patents for over 30 years and can say with certainty, this patent and a series of others are pointing to a massive shift for Siri, Messages App and the new #VoiceFirst system I have been calling HomeBase.



I dive in to the way this patent will change the direction of Apple in a number of ways and present the opportunities it will create for startups and legacy companies.  The Multiplex Memorandum is a feature of the Multiplex Magazine subscription.  You are on the web based blog for the Multiplex App. This plus much more can be accessed exclusively through the Multiplex app found only on the iOS app store.  Though the Multiplex app you can choose to subscribe.  Subscriber features include:

  • News Flash Breaking news and notifications
  • Regular issues of Multiplex Magazine
  • Back issues of Multiplex Magazine
  • Multiplex Memorandum
  • Multiplex slide decks
  • Access to lower cost consulting and advisement
  • And much more…

Subscribe Today! 




Subscribe ($99) or donate by Bitcoin.

Copy address: bc1q9dsdl4auaj80sduaex3vha880cxjzgavwut5l2

Send your receipt to to confirm subscription.

Subscribe to this site, This is not for paid site membership or to access member content. To become a member, please choose "Join Us" on the main menu.


IMPORTANT: Any reproduction, copying, or redistribution, in whole or in part, is prohibited without written permission from the publisher. Information contained herein is obtained from sources believed to be reliable, but its accuracy cannot be guaranteed. We are not financial advisors, nor do we give personalized financial advice. The opinions expressed herein are those of the publisher and are subject to change without notice. It may become outdated, and there is no obligation to update any such information. Recommendations should be made only after consulting with your advisor and only after reviewing the prospectus or financial statements of any company in question. You shouldn’t make any decision based solely on what you read here. Postings here are intended for informational purposes only. The information provided here is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of your physician or other qualified healthcare provider with any questions you may have regarding a medical condition. Information here does not endorse any specific tests, products, procedures, opinions, or other information that may be mentioned on this site. Reliance on any information provided, employees, others appearing on this site at the invitation of this site, or other visitors to this site is solely at your own risk.

Copyright Notice:

All content on this website, including text, images, graphics, and other media, is the property of Read Multiplex or its respective owners and is protected by international copyright laws. We make every effort to ensure that all content used on this website is either original or used with proper permission and attribution when available. However, if you believe that any content on this website infringes upon your copyright, please contact us immediately using our 'Reach Out' link in the menu. We will promptly remove any infringing material upon verification of your claim. Please note that we are not responsible for any copyright infringement that may occur as a result of user-generated content or third-party links on this website. Thank you for respecting our intellectual property rights.