Tech Buzz: Zuckerberg’s own ‘Jarvis’

Taking a leaf out of Iron Man Tony Stark’s book, Mark Zuckerberg has developed a virtual assistant for his home. It’s called Jarvis (of course), and it responds to text and voice commands which could involve asking it to play music, turn on the air conditioning, open doors, and even recognize visitors. Jarvis understands context, for better response to requests such as personal tastes in music.

Incidentally, Jarvis speaks in actor Morgan Freeman’s voice.

First things first: Zuckerberg had to ensure that all the existing gadgets in his home were connected to the Internet. This included a Crestron system for lights, thermostat and doors, a Sonos music system with Spotify, a Samsung TV and a Nest camera. Since all these gadgets work on different protocols, it was important to make them talk to each other, even as they were connected to the Internet.

The next step was to write a code which would understand keywords and, subsequently, the spoken word. In this case, there needed to be variations in the task to be done, depending on who was uttering the command, and that contextual understanding was important in a home environment. “The more context an AI (artificial intelligence) has, the better it can handle open-ended requests. At this point, I mostly just ask Jarvis to ‘play me some music’ and by looking at my past listening patterns, it mostly nails something I’d want to hear. If it gets the mood wrong, I can just tell it, for example, ‘that’s not light, play something light’, and it can both learn the classification for that song and adjust immediately," Zuckerberg said in a blog post.

Then there is the matter of face recognition. While choosing between two objects is simpler, identifying two potentially similar faces may be a challenge. But Facebook is already quite good at identifying your friends, and that helps with Jarvis as well.

It isn’t a surprise that Zuckerberg is using the Messenger app to communicate, which allows Zuckerberg to communicate and command Jarvis from anywhere, via text.

This is just the start, because AI still needs a lot of work. At the same time, Zuckerberg predicts that in about 5-10 years, AI systems will be more accurate than people for each of our senses—vision, hearing and touch, as well as language.