AS101: Engineering Exploration

On

PROJECT
VIVA
Department of Computer Science and Engineering,
Chitkara University, Punjab

Name: Tarush Malhotra

Roll No.: 1810991089
Intelligent
Virtual
Assistant
What is an IVA?
According to Wikipedia, an IVA is “an intelligent virtual
assistant (IVA) or intelligent personal assistant (IPA) is
a software agent that can perform tasks or services for an
individual based on commands or questions. The term
"chatbot" is sometimes used to refer to virtual assistants
generally or specifically accessed by online chat. In some
cases, online chat programs are exclusively for
entertainment purposes. Some virtual assistants are able to
interpret human speech and respond via synthesized voices.
Users can ask their assistants questions, control home
automation devices and media playback via voice, and
manage other basic tasks such as email, to-do lists, and
calendars with verbal commands.”
An IVA is a combination of:
• Automatic Speech Recognition
• Artificial Intelligence
• Natural Language Processing
• Inter-process Communication

It is basically an assistant (someone who does your tasks),

but is virtual, and yes, not living.
 History
• The concept of a virtual assistant was developed by
Joseph Weizenbaum of MIT in the 60s
• He developed the first chatbot “Eliza”
• While the first PDA was released in 1984 by Psion
• Early PDAs were devices with full keyboard and a touch
display
• These PDAs were based on the OS known as PALM
The current generation of IVAs
• The current generation of IVAs can do a lot of
tasks. And all a user needs to do is say their ‘call’
name, dictate their command, and the assistant
will do its work.
• They can call someone on the user’s phone, text
someone on the user’s phone using a particular
application if requested by the user, play a song
told by the user, play any album of any artist,
email someone something, and the list is just
infinite.
• The most famous Virtual Assistants in the current
generation of IVAs are – Google Assistant, Siri, Cortana,
and Alexa.
• Each of the assistants work in a similar way – the user
says each assistant’s ‘call’ name, the assistant activates,
the user tells what they want the assistant to do, and the
assistant does that.
• But each assistant’s capabilities and features are quite
varied – one assistant is more ‘intelligent’ than the others
in one sphere of information retrieval and the way it is
given to the user, while one assistant is more ‘intelligent’
than its counterparts in some other sphere.
• While each of the assistant is more useful in its own ways,
around 85% of the researchers on the web have called
the Google assistant to be the overall best among them.
Google Assistant
• Google Assistant is the virtual assistant developed by
Google that is primarily available on mobile phones and
smart home devices.
• This assistant can engage in two-way conversations.
• Its initial release was in May 2016 as a part of the
company’s messaging application Allo, and its voice-
activated speaker Google Home.
• After a period of exclusivity on the Pixel smartphones, it
began to be deployed on other Android devices in
February 2017, including third-party smartphones and
Android Wear, and was released as a standalone app on
the iOS in May 2017.
• The Assistant has been further extended to support a
large variety of devices, including cars and third-party
smart home appliances.
• It can search the Internet, schedule events and alarms,
adjust hardware settings on the user's device, and show
information from the user's Google account.
• Google has also announced that the Assistant will be able
to identify objects and gather visual information through
the device's camera, and support purchasing products
and sending money.
• It is used by more than half a billion people on a daily
basis.
• PC World's Mark Hachman gave a favorable review of the
Google Assistant, saying that it was a "step up on Cortana
and Siri."
Siri
• Siri is a virtual assistant that was developed by Apple Inc.
• It was released in October 2011, when it was first
integrated into the company’s then latest iPhone – the
iPhone 4s.
• The assistant uses voice queries, gesture-based control,
focus-tracking and a natural-language user interface to
answer questions, make recommendations, and perform
actions by delegating requests to a set of Internet
services.
• The software adapts to users' individual language
usages, searches, and preferences, with continuing use.
Returned results are individualized when using the
assistant over a longer period of use.
• It is part of Apple’s iOS. It is used in all the products of
Apple – iPhone, iPod, iPad, iMac, MacBook, iWatch,
Apple TV, and HomePod.
• Siri, just as Google Assistant can perform a wide-range of
actions as calling, facetiming, texting, or emailing
someone, checking the basic information like ‘today’s
weather’, finding basic facts.
• The recent updates of Siri has made it able for the user to
ask it to even handle payments through their payment
application, Apple Pay.
• After the 2015 update of Siri, it trains to just identify the
phone owner’s voice, in order to prevent non-owner
activation.
• Since the iOS 11, it can handle follow-up questions,
supports language transition, and do task management.
Cortana
• Cortana is a virtual assistant which was
developed by Microsoft.
• It was released in April 2014, when it was
unveiled for the first time at Microsoft Build
Developer conference in San Francisco.
• Its development begun in the year 2009 in the
company’s Speech product team and was
headed by Zig Serafin and Larry Heck.
• For making it as much life-like as possible, they
interviewed over 250 human personal assistants.
• In January 2015, the initial deployment of the
assistant began.
• It was first introduced into the laptops running
Windows 10, and soon after into the Windows
phone.
• Then in May 2015, the company announced its
deployment into other mobile platforms.
• It was released on Android and on iOS in December
2015. But the company decided to shut down the
Cortana Mobile app globally in the end of March
2021.
• Even on the most recent Windows update of
Windows 11, the company has reduced the
emphasis on it. It has been removed from the
taskbar and is now not used during the new device
Echo
• Alexa is a virtual assistant technology developed by
Amazon.
• It was released in November 2014, when the company
announced Alexa along with the smart speaker Echo.
• Alexa is now the most capable (not ‘intelligent’) virtual
assistant, in the sense that it is the most integrated
assistant into the smart devices.
• It thus, uses itself as home-automated systems.
• Now, more developers are developing skills for Alexa than
all the other assistants combined. This was possible due
to the company’s open development initiative.
• According to the company’s featuring, as of November 2018,
more than 10,000 employees were working on Alexa and its
related products.
• Based on the system of Alexa, the company has released a
lot of groundbreaking products – Echo Studio, the first smart
speaker with 360 sound and Dolby Surround, Echo buds,
Alexa built-in wireless earphones, Echo Frames, Alexa built-
in spectacles, Echo Loop, Alexa built-in ring.
• In January 2019, the Amazon’s devices development team
announced that they had sold over 100 million Alexa-enabled
devices.
• It has support for more than 15 sport leagues, which any
Alexa enabled device user can ask it to tell the score about.
• The most recent innovative development in the Alexa’s
interface is that the user can train the Alexa device to
recognize each of their family member’s voice separately.
Future of IVAs
• In May 2018, Google revealed Duplex, an extension of the
Google Assistant that allows it to carry out natural conversations
by mimicking human voice, in a manner not dissimilar to
robocalling.
• The assistant can autonomously complete tasks such as calling
a hair salon to book an appointment, scheduling a restaurant
reservation, or calling businesses to verify holiday store hours.
• While Duplex can complete most of its tasks fully autonomously,
it is able to recognize situations that it is unable to complete and
can signal a human operator to finish the task.
• Duplex was created to speak in a more natural voice and
language by incorporating speech disfluencies such as filler
words like "hmm" and "uh" and using common phrases such as
"mhm" and "gotcha", along with more human-like intonation and
response latency
• Duplex is currently in development and had a limited
release in late 2018 for Google Pixel users.
• During the limited release, Pixel phone users in
Atlanta, New York, Phoenix, and San Francisco were
only able to use Duplex to make restaurant
reservations.
• As of October 2020, Google has expanded Duplex to
businesses in eight countries.
• This new venture of Google is really going to be a
game-changer in the market.
• Imagine wanting to call your doctor for scheduling an
appointment. And instead of waiting for your turn to
come in as your doctor is quite famous, the Duplex
will do the work for you. It is going to do the
scheduling for you
Concerns regarding IVAs
• The biggest concern that the Privacy Advocates
across the globe raise is the amount of voice
samples sent to the assistant’s backend (which
in turn is given to the company as that is the
place where it is stored), though making them
smarter after each run, but compromising the
user’s privacy at its very own core.
• Albeit these features individualize the user
experience, they are unsure about the long-term
implications of giving "the company
unprecedented access to human patterns and
preferences that are crucial to the next phase of
• A few years back, a Belgian public broadcaster
published an article revealing that third-party
contractors paid to transcribe audio clips
collected by certain Virtual Assistants listened to
sensitive information about users.
• From over 1000 recordings analyzed, 153 were
recorded without the “Okay, Google” command,
which was later on confirmed by the company
that the same was did to improve Google’s
services.
• They officially acknowledged that around 0.2%
were recorded without consent, of which
consisted private conversations.
• In July 2019, an anonymous whistleblower of
Apple said that Siri regularly records some of its
users' conversations even when it was not
activated. The recordings are sent to Apple
contractors grading Siri's responses on a variety
of factors. Among other things, the contractors
regularly hear private conversations between
doctors and patients, private business, and
everything else too.
• Then in August 2019, the company took the
blame, apologized and said that it is going to halt
the Siri grading program and delete all the stored
voice recordings. But still, no solid response has
been released by the company since then.
• So, there is comfort in the use of the Virtual
Assistants, there is bad in the use of the Virtual
Assistants.
• If we are so keen on using the Virtual Assistants,
then we must pay the before mentioned ‘tax’ and sell
our privacy for free to them.
• But if we do not want such thing, then we surely can
take the harder route – throwing away our
smartphones in the trash and using the button
phones instead. And yeah, it is hard.
• But either way, it is not a choice we have. Our
privacy is intruded some way, and we cannot run
away from that. So, what we can do is show as little
of ourselves as possible and stay away from all the
possible privacy-breakers as much we can.