Journal of Retailing and Consumer Services 61 (2021) 102551

Contents lists available at ScienceDirect

Journal of Retailing and Consumer Services

journal homepage: http://www.elsevier.com/locate/jretconser

The future of service: The power of emotion in human-robot interaction

Stephanie Hui-Wen Chuah a, Joanne Yu b, *
a
Institute of Innovation and Circular Economy & Department of Business Administration, Asia University, No. 500, Liufeng Rd., Wufeng Dist., Taichung City, 413,
Taiwan
b
Department of Innovation and Management in Tourism, Salzburg University of Applied Sciences, Urstein Süd 1, A-5412, Puch, Salzburg, Austria

A R T I C L E I N F O A B S T R A C T

Keywords: Astoundingly, recent technological advancements have enabled robots to display emotions. Yet, while emotional
Human-robot interaction expression is valued in the field of service, understanding emotions in human-robot interaction remains
Emotional contagion underexplored. Since emotions are contagious/transmittable, this study utilised Instagram data to uncover how
Machine learning
emotional robots influence potential consumers’ affective feelings. By employing machine learning algorithms
Sentiment
Facial recognition
and sentiment analysis, the findings suggest that the expressions of surprise and happiness are key to creating
Consumer experience positive impacts on potential consumers. The cross-disciplinary nature of this study lays the groundwork for next-
Service robot level social, design, and creative experiences in artificial intelligence research regarding consumer service and
experience contexts.

1. Introduction emotional contagion (Hatfield et al., 2014), which has been

well-documented in human-human interactions (Rymarczyk et al.,
The integration of artificial intelligence in customer service and 2019; Woo and Chan, 2020). In such an interaction, one tends to sub­
experience has been one of the most exciting yet disruptive practices to consciously mimic the facial expression displayed by another individual
date. With the ongoing worldwide health crisis, one cannot deny the fact or interlocutor (Otterbring, 2017). This is especially salient in customer
that service automation is shifting the paradigm from high-touch to experiences, highlighting direct interactions between consumers and
high-tech (Zeng et al., 2020). The use of robots has increasingly emerged service agents (Söderlund, 2017). In particular, previous studies have
in retail, airports, hotels, restaurants, and transportation areas, amongst underpinned the interplay between emotions and service recovery
others (Prentice and Nguyen, 2020; Zeng et al., 2020). Based on their (Ngan and Yu, 2019), consumer satisfaction (Söderlund, 2017), and
appearance, robots adopted in service environments can be roughly customer empathy (Ngo et al., 2020).
classified as a combination of functional, cartoonish, and humanlike, or Advanced technologies in artificial intelligence and machine
mixed robots (Tung and Au, 2018). With a particular focus on humanoid learning have revolutionised the nature of consumer experiences by
robots that can anthropomorphise human behaviour most efficiently enabling robots to support services. For instance, Hilton Worldwide has
(Christou et al., 2020), recent studies have investigated how their voice employed a service robot, Connie, as the concierge (Lu et al., 2019),
appearance (Yu, 2020b), voice (Murphy et al., 2019), and facial ex­ while Japan’s Kinosaki Onsen hot spring resort has adopted an inter­
pressions (Yu and Ngan, 2019) act upon consumers during service en­ active humanoid robot to communicate with tourists in the information
counters. In fact, these nonverbal cues are of core importance in any centre (Frommer, 2015). More recently, a Hong Kong-based company
face-to-face interaction since they support and add additional meaning developed Sophia, the first anthropomorphic robot capable of displaying
to verbal communication (Pantano, 2020). Among various nonverbal more than 60 different emotions (Faraj et al., 2020). Alongside the
signals, facial expressions form a universal language by presenting COVID-19 pandemic, acting as a driving force behind robotisation in the
emotional states and feelings (Gendron et al., 2018). Through the acti­ field of general service, businesses have envisioned robots as a means to
vation of facial muscles, humans can express emotions including anger, help facilitate operations during this new period of “normality” (Seyi­
disgust, fear, happiness, sadness, and surprise (Gendron et al., 2018). toğlu & Ivanov, 2020). However, apart from the fact that practitioners
Notably, emotions are also known to be contagious and can be trans­ may still be unprepared for the integration of robotics (Ivanov et al.,
mitted to others (Torre et al., 2020). This notion is referred to as 2020), service employees also have concerns surrounding potential job

* Corresponding author.
E-mail addresses: hw.chuah@gmail.com (S.H.-W. Chuah), joen841030@gmail.com (J. Yu).

https://doi.org/10.1016/j.jretconser.2021.102551
Received 27 January 2021; Received in revised form 6 March 2021; Accepted 26 March 2021
Available online 7 April 2021
0969-6989/© 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

replacements (Io and Lee, 2020). While some suggest that humanoid might begin copying another individual’s gestures or facial expressions.
robots are preferred by consumers compared to other types (Christou This notion of mimicry is often spontaneous and helps humans empa­
et al., 2020), other findings have revealed contradictory results (Mende thise with others by simulating their feelings (Rymarczyk et al., 2019).
et al., 2019). Thereafter, the feedback stage initiates, referring to when one starts to
In that vein, consumers’ negative feelings (e.g. fear, anger, and truly understand the feeling portrayed by another (Woo and Chan,
sadness) during human-robot interaction should not be overlooked 2020). As a result of this feedback, the emotion of others is then trans­
(Christou et al., 2020) since they could lead to subsequent negative re­ formed into one’s own experience (Hatfield et al., 2014), and the feeling
actions. Yet, the notion of whether emotional contagion can be extended becomes prominent.
to human-robot interaction remains largely unclear. Since displaying Previous studies have long recognised the importance of this series of
emotions increases the degree of human likeness and stimulates actions. For instance, one study highlighted the relationship between the
emotional bonds between users and robots (Rincon et al., 2019), it emotions of employees and consumers at the front desk (Ustrov et al.,
signifies the need to disclose the emotional transmission in human-robot 2016). Similarly, Otterbring (2017) demonstrated that the expression of
communication (Stock, 2016; Yu and Ngan, 2019). Leveraging con­ happiness and joy displayed by employees increases the level of con­
sumer emotions in the context of human-robot services is beneficial to sumer pleasure during service encounters. Nonetheless, given the
value co-creation (Neuhofer et al., 2020b), satisfaction (Yu and Ngan, multifaceted nature of emotions, knowledge related to negative affec­
2019), and acceptance towards new technologies (Fuentes-Moraleda tivity in service experiences remains under-researched and merits
et al., 2020). Although one can self-report his/her own feelings towards further attention (Liu et al., 2019). Liu and Liu (2020) claim that
robots (Christou et al., 2020), advanced data analytics has enabled re­ negative emotions presented by service agents cannot be transmitted to
searchers to take a step forward by assessing one’s emotions through consumers potentially because employees tend to hold back negative
natural language processing, machine learning (Chatterjee et al., 2019), facial expressions in service contexts. As such, Liu et al. (2019) discov­
and facial recognition technology (Yu, 2020a). ered that the contagion of negative feelings is more obvious in relation
As such, this research aims to investigate how robots with emotions to surface acting, the superficial display of emotions. Notably, however,
influence potential consumers’ affective states. By using machine interactions in contemporary service contexts are no longer limited to
learning algorithms and sentiment analysis techniques based on Insta­ humans. Instead, the emergence of service robots, along with anthro­
gram data, this study examines the extent to which emotions can in­ pomorphic robots as well, underlines the significance of human-robot
fluence viewers’ feelings. This study is novel in that it bridges and interaction in the field of service (Choi et al., 2020).
incorporates the disciplines of psychology and data science into the
context of human-robot interaction within the domain of service 2.2. The role of emotion in human-robot interaction
research. Indeed, despite the full potential of robots’ emotional capacity
still remaining far beyond reach, this research calls for the need to look Essentially, human-robot interaction is considered multidisciplinary
beyond the status quo and uncover the possible effects of emotional in nature, involving the fields of psychology, social sciences, cognitive
intelligence in relation to foreseeable service ecosystems. By zooming in science, artificial intelligence, design, engineering, and computer sci­
on what may likely transform the future of experiences, the emotional ence (Joe and Song, 2019). The study of robots in service environments
impact between robots and potential consumers, this study lays the expands the knowledge of consumers’ experiences (Neuhofer et al.,
groundwork for service businesses that intend to adopt anthropomor­ 2020a) and paves the way for potential applications of service robots
phic and/or humanoid robots. It also offers insights into and opens up (Choi et al., 2020). Notwithstanding the ongoing debates and contro­
several avenues of investigation for the design and application of versial issues related to safety and security (Ivanov et al., 2020; Mende
anthropomorphic robots. et al., 2019), some stakeholders in the service industries have envisioned
anthropomorphic/humanoid robots as the next contemporary social
2. Literature review practice (Choi et al., 2020). On that note, a number of studies in
particular have hinted at emotions counting as the most crucial factor in
2.1. Emotional contagion: the what and how human-robot interaction (Shank et al., 2019; Yu and Ngan, 2019) since
they facilitate engagement and co-create experiences (Rincon et al.,
Facial expressions are critical for any form of social communication 2019). Torre et al. (2020), for instance, revealed that consumers present
between humans (Ngan and Yu, 2019), and some emotional expressions, a higher degree of trustworthiness towards robots displaying positive
such as anger, contempt, disgust, fear, joy, sadness, and surprise, have emotions. Likewise, Appel et al. (2020) pointed out that robots with
been noted across cultures as universal concepts (Gendron et al., 2018). emotions are preferred over a neutral counterpart. This notion is closely
These feelings and emotions serve as the cornerstone of all experiences related to the theory of uncanny valley, suggesting the relationship be­
(Gohary et al., 2016) as they facilitate cognitive processes and help in­ tween human-likeness and emotions (Appel et al., 2020; Mathur et al.,
dividuals make sense of the world (Isen, 2009). Moreover, human af­ 2020). However, humanoid robots that imperfectly resemble actual
fective states are influential towards one’s motives and behaviours (Isen, human characteristics could provoke negative emotional responses in
2009). One particular reason being that emotions can be transmitted observers (Mathur et al., 2020). Although the significance of subtle
(Torre et al., 2020; Woo and Chan, 2020). In this way, individuals can be emotional attributes has been established, only more recently have
infected by someone else’s happiness or sadness and vice versa, working scholars started to investigate the interchange between emotions/­
as a reciprocal process. Grounded in the discipline of psychology, this feelings presented by both robots and humans (Mathur et al., 2020;
phenomenon is referred to as emotional contagion (Hatfield et al., 2014) Shank et al., 2019; Torre et al., 2020).
and reflects the fact that humans, either consciously or unconsciously, Closely related to robotics, earlier research has underpinned the fact
mimic the emotion expressed by people around them (Prentice, 2016). that adding emotional elements to human-computer interactions can
Neuroscientists suggest the mirror neuron system as a possible expla­ enrich conversation (Zhou et al., 2017). If chatbots deliver messages
nation behind the dynamics of emotional contagion (Decety and Ickes, with a joyful tone, users typically experience more positive and happy
2011). In fact, transferring emotions not only exists in humans but also feelings of emotion (Zhou et al., 2017). Other scholars have also looked
occurs in other non-human primates, for example, dogs with their body into the simple emotions expressed by the Nao robot as front-line service
postures (Palagi et al., 2015). agents, suggesting that emotional contagion is more likely to occur with
Broadly speaking, the process of emotional contagion involves three positive emotions rather than negative ones (Stock, 2016). Yet, Raffard
stages; namely, mimicry, feedback, and contagion (Hatfield et al., et al. (2016) found opposite results in the context of clinical services in
2014). As soon as an emotion has been (unconsciously) recognised, one which negative expressions (e.g., anger) were recognised quicker in

2
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

comparison to positive emotions (e.g., happiness). These contradictory robots can mimic human facial expressions is required (Wirtz et al.,
findings may result from the nature of the research field since users 2018), understanding the effect of emotional contagion in human-robot
might be more sensitive towards negative emotional expressions within interaction can, without a doubt, add empathy and provide more per­
a hospital environment (Raffard et al., 2016). Thus, to further improve sonalised service experiences in the future (Mich, 2020).
trust and cooperation between humans and robots in healthcare ser­
vices, researchers developed a robot capable of expressing empathy in 3. Methodology
accordance with users’ emotional states. This is achieved through the
robot’s movements of its eyebrows and mouth to present seven different Overall, this research aims to examine the interplay between the
emotions (Rincon et al., 2019). emotional expressions produced by robots and consumers’ affective
Overall, despite existing literature revealing some interesting in­ responses using machine learning methods. This study measured robotic
sights into robots with emotions, the study subjects only resemble the emotional expressions in terms of anger, contempt, disgust, fear,
shape of humans (e.g., Nao and Pepper) rather than human physical happiness, neutral, sadness, and surprise, whereas the consumers’ af­
appearance. With the rise of artificial intelligence, Henn-na hotel in fective reaction was measured by positive, negative, and neutral senti­
Japan, which used to be fully staffed by robots, has further employed ments. Sophia, the first anthropomorphic robot to express over 60
anthropomorphic robots as front desk agents (Ivanov et al., 2018). A human feelings/emotions, was selected as the study context (Faraj et al.,
study by Yu and Ngan (2019) initiated experimental research by 2020). Sophia was developed by Hanson Robotics in Hong Kong and was
manipulating the smiling behaviour of anthropomorphic robots at even granted Saudi Arabian citizenship in 2017 (Hanson Robotics,
Henn-na hotel. In particular, positive emotional displays led to a higher 2020). Technically speaking, Sophia relies on deep learning and
level of perceived service quality (Yu, 2018), interpersonal warmth, and advanced material technology to simulate the major facial muscles of
customer satisfaction (Yu and Ngan, 2019). Indeed, although it remains humans, allowing it to present joy, confusion, surprise, and sorrow,
difficult to capture the full potential of robots with emotions given the amongst others. Nonetheless, it is important to note that despite the
lack of real interaction, consumers generally expect artificial intelli­ ultimate goal of Sophia being able to serve in healthcare, customer
gence to be able to process human emotions so as to provide personal­ service, and education (Hanson Robotics, 2020), Sophia has yet to be
ised offers during experiences (Grundner and Neuhofer, 2021). employed in the service industries. In this research, the methodological
procedure involved six distinct steps, which will be presented in more
2.3. A new paradigm in human-robot research detail below.

While the importance of emotions and facial expressions has been 3.1. Step 1: Date extraction
acknowledged, little research has quantified and determined different
emotions in human-robot interaction in a data-driven and systematic To identify pictures of Sophia for the subsequent analysis, this study
way. Derived from the discipline of data science, big data has provided a extracted photos published on Sophia’s official Instagram account
sea change regarding how knowledge can be constructed and research (@realsophiarobot: https://www.instagram.com/realsophiarobot/).
can be conducted (Balazka and Rodighiero, 2020). In fact, it is the Data extraction was conducted in February 2020 using Octoparse, a web
explorative nature of big data that allows researchers to transform un­ scraping tool that has increasingly been applied to service and market­
structured datasets into meaningful and interpretable outcomes (Arora ing contexts (Lu et al., 2021; Yu et al., 2020). Based on a total of 391
et al., 2019). Emerging data analysis techniques in marketing and posts available at that time, the links of posts and images, the type of
management include, but are not limited to, machine learning, artificial posts (i.e., pictures/videos), and the captions were crawled. Meanwhile,
intelligence, natural language processing, and data mining. In relation to corresponding comments for each of the posts, timestamps, and user­
this particular focus of study, analysing emotions/feelings can best be names were also extracted. This resulted in a total of 8,989 comments
achieved through the means of natural language processing (e.g., left by 4,308 Instagram users.
sentiment analysis) when it comes to texts (Celuch, 2020) and facial
recognition technology in the case of pictures/videos (Nie, 2020). More 3.2. Step 2: Data cleaning
specifically, sentiment analysis quantifies the valence of a sentence
based on the provided text (Celuch, 2020). Under the guise of natural To analyse Sophia’s emotional expressions, this study focused
language processing, detecting the polarity of textual information pro­ explicitly on images. Thus, 42 video-posts and their corresponding
vides additional value regarding consumers’ feelings towards service comments were removed. Thereafter, the researchers manually sifted
robots (Fuentes-Moraleda et al., 2020), uncovers users’ affective states through the data in order to remove pictures with “cartoonish” char­
when communicating with robots (Atzeni and Recupero, 2020), and acteristics of Sophia (e.g., Little Sophia: https://www.instagram.com
optimises reactive emotions to improve human-robot interaction (Li /p/BtPD_iFHL5H/). Next, the remaining posts were imported into a
et al., 2017). facial recognition API developed by Microsoft Azure. This is an artificial
In addition to text data, facial recognition systems, a method of intelligence service that detects the faces and attributes of an image (e.
biometric identification, identify and analyse human faces based on g., Chen et al., 2020) and returns image annotation, including a range of
digital images or videos (Nie, 2020; Pantano, 2020). More service facial expressions. Pictures in which a face was not recognised were
businesses have implemented such technology in order to better un­ excluded. This may have occurred due to Sophia being too small or
derstand consumer experiences (Pantano, 2020). In human-robot looking back in the pictures. Therefore, the cleaned dataset resulted in
interaction, facial recognition has mainly been installed in robots so as 223 posts with 6,041 comments left by 4,308 Instagram users.
to detect the emotions of users for the purpose of entertainment
(Hasegawa and Takahashi, 2019) and security (Nie, 2020). Additionally, 3.3. Step 3: Emotional recognition of Sophia
with improvements in artificial intelligence, researchers have embraced
the potentials of cloud-based services such as the emotion/facial Sophia’s emotional expressions were identified by the facial recog­
recognition application programming interfaces (API) provided by nition API, as introduced in the previous section. It is important to note
Microsoft, Google, and Amazon (Pantano, 2020). For instance, Chen that in the pictures containing several people, only Sophia’s emotions
et al. (2020) adopted Microsoft Azure to detect the emotional states of were detected. The links to the pictures were first imported to Microsoft
consumers based on photographs, while a Japan railway company uses Azure, which returned the facial attributes of anger, contempt, disgust,
IBM Watson to analyse customers’ tone and emotion (Mich, 2020). fear, happiness, neutral, sadness, and surprise. The score, ranging from
Regardless of the fact that further investigation on the extent to which 0 to 1, implies the strength of a specific emotion. In this case, a higher

3
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

value indicates a stronger emotional expression displayed in the picture. could contribute the most to the quality of the machine learning models.
In order to find the best features for machine learning (Yu and Liu,
3.4. Step 4: text pre-processing 2003), surprise, happiness, neutral, and sadness were included in the
learning process. The other emotions (fear, disgust, contempt, and
The 6,041 comments identified in step 2 were pre-processed for the anger) were excluded since nearly all values were identical (i.e., closer
following sentiment analysis so as to uncover consumers’ affective re­ to 0). Subsequently, this study made use of the automatic optimisation
actions. First, non-English comments and unknown characters, URLs, feature in RapidMiner. This algorithm optimises (1) the maximal depth
numbers, and references to usernames (i.e., @username) were removed. for decision tree, (2) the number of trees and maximal depth for random
Additionally, words followed by a hashtag (#) were eliminated as well. forest, (3) the number of trees, maximal depth, and learning rate for
Next, the researchers prepared a list of stopwords to remove irrelevant gradient boosted trees, and (4) the gamma and C hyperparameter for
terms (e.g., ‘and’, ‘you’, ‘I’). These procedures resulted in a total of 3,777 support vector machine.
remaining comments, which were then transformed into lowercase. In Similar to recent studies in the field of management/marketing (e.g.,
order to reduce morphological variation, comments were lemmatised Biyik and Tanyeri, 2018; Nikou et al., 2019; Shafqat and Byun, 2020),
and stemmed using Wordnet Lemmatizer and Porter Stemmer, respec­ this research evaluated the performance of the models based on the
tively. Finally, comments were tokenised into small units for the appli­ mean squared error (MSE) and the root mean squared error (RMSE). The
cation of sentiment analysis. former is suitable for datasets with outliers and unexpected values, while
the latter is useful for datasets containing large errors (Willmott, 1982).
3.5. Step 5: Sentiment analysis It is relevant to bear in mind that although there is no rule of thumb, a
lower value technically indicates better performance. Finally, the algo­
To detect the affective states embedded in the comments, this study rithms highlighted the most influential emotional features towards the
employed a lexicon-based sentiment analysis with the incorporation of compound sentiment based on weight vectors (Ribeiro et al., 2016). It
the Vader algorithm. The Vader algorithm was chosen based on its ad­ then visualised and predicted the strength of the different emotions that
vantages in analysing emojis, slangs, and acronyms commonly used by could provide the best learning models.
social media users (Celuch, 2020). It also outperforms other sentiment
analysis techniques for social media contexts because Vader is built on a 4. Results
valence-based sentiment lexicon in accordance with Natural Language
Toolkit (Gilbert and Hutto, 2014). Technically speaking, Vader returns The facial recognition API categorised one’s emotional expressions
four types of sentiments; namely, positive, negative, neutral, and com­ into seven different states: anger, contempt, disgust, fear, happiness,
pound values. The compound score suggests a single measure of polar­ neutral, sadness, and surprise. Table 1 provides an overview of the in­
ity, ranging from − 1 (the most negative) to +1 (the most positive), tensity of these emotions. Based on Instagram pictures, Sophia’s emo­
whereas a neutral feeling equates to 0. The compound value, on the tions are dominated by neutral and happiness, followed by sadness and
other hand, is a normalised score of the sum of computed valence values surprise. Regarding the valence of the comments in response to Sophia’s
based on the sentiment lexicon (Costa et al., 2019). Similar to other posts, the findings suggest that Instagram users’ feelings towards Sophia
studies implying its usefulness (e.g., Costa et al., 2019; Lundqvist et al., have a relatively low value. Furthermore, based on the overall com­
2020), the following analysis was also based on the compound score. pound scores, a relatively low positive experience towards Sophia was
found, with a sentiment value of 0.2 (Table 2). Thus, this opens up an
3.6. Step 6: Implementation of machine learning interesting avenue for investigation regarding how specific emotions
can be transmitted in human-robot interactions.
Lastly, in order to analyse how Sophia’s emotional expressions act on Four machine learning algorithms were performed in this study to
consumers, machine learning techniques were implemented using the examine the relationship between robot and human emotions. This
Auto Model Tool available in RapidMiner. RapidMiner is an open-source study took MSE and RMSE as the performance metrics for evaluation,
platform for data mining, machine learning, and predictive modelling, which are provided in Table 3. Consistent with previous literature
and has been previously applied in research relating to consumer ex­ (Ebrahimi et al., 2019; Gong et al., 2019), the results demonstrate a
periences (Buzova et al., 2019; Yan et al., 2018). Overall, on account of higher accuracy score of gradient boosted trees. To be precise, the
this study’s datasets containing nonlinear structure, machine learning optimal parameters of the gradient boosted trees identified in this study
methods are capable of detecting possible nonlinear relationships and consisted of 15 for number of trees, 2 for maximal depth, and 0.100 for
higher-order interactions (Ryo and Rillig, 2017). Distinct from conven­ learning rate. Consequently, the gradient boosting results were chosen
tional statistical modelling (e.g., regression analysis), machine learning for the subsequent data interpretation. Nonetheless, bearing in mind
algorithms stand out on the premise that prior assumptions are not that each algorithm has its pros and cons, one of the epistemological
required for the datasets (Recknagel, 2001). challenges of big data research involves the accuracy of the algorithms
In order to ensure the reliability of the results, the relative perfor­ as well as the interpretability of the results (Elragal and Klischewski,
mance of four different methods commonly used in supervised learning 2017). Although this study selected the algorithm with the best per­
for regression (i.e., decision tree, random forest, gradient boosted trees, formance, the reality of big data, meaning that knowledge production
and support vector machine) were compared. The first three techniques and interpretation are often subject to human bias, needs to be taken
entail tree-based algorithms (Fan et al., 2018). They are non-parametric into account (Balazka and Rodighiero, 2020; Symons and Alvarado,
in nature and are less affected by outliers and multicollinearity. In this 2016). While acknowledging that epistemological pitfalls warrant
regard, decision trees involve a series of sequential steps and produce a further investigation, these are considered beyond the scope of this
simple decision-making diagram (Ogutu et al., 2011). Random forests study.
combine several decision trees at the end of the process, whereas The learning model produced by the gradient boosting algorithm
gradient boosting combines tress at the beginning of the process (Ogutu returned the local weights of Sophia’s emotional expressions (see
et al., 2011). Contrarily, support vector machine employs Table 4). These weights in particular suggest which emotions are
distance-based algorithms and is more effective and accurate when a generally most influential in relation to the valence of Instagram com­
clear margin of separation is present (Fan et al., 2018). ments. The results demonstrate that surprise is the most prominent input
First, the compound sentiment scores, described in section 3.5, were feature affecting Instagram users’ affective reactions towards Sophia’s
assigned as the target variables. Subsequently, feature selection of the images. This is followed by the expressions of happiness, neutral, and
input variables was conducted based on emotional expressions that sadness.

4
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

Table 1
Overview of emotional expressions.
Anger Contempt Disgust Fear Happiness Neutral Sadness Surprise

Mean 0.001 0.003 0.001 0.010 0.325 0.513 0.079 0.071

SD 0.002 0.006 0.002 0.034 0.363 0.361 0.147 0.164

Note: the strength ranges from 0 to 1.

score, valued at 0.571.

Table 2
Overview of sentiment from Instagram comments.
5. Discussion
Positive Negative Neutral Compound

Mean 0.254 - 0.051 0.695 0.227 As the service industry goes hand in hand with technological im­
SD 0.309 - 0.145 0.313 0.387 provements, it is critical for scholars and practitioners to maintain a
Note: the sentiment ranges from − 1 to +1. futuristic mindset in order to envision and predict emerging practices in
service environments as well as advance and picture promising con­
sumer experiences. Although service robots with emotional intelligence
Table 3 are continuing to progress and emerge, this is merely to a limited extent.
Performance of machine learning. Nonetheless, Grunder and Neuhofer (2021) highlight the necessity to go
Techniques Mean squared error Root mean squared error beyond the status quo and look towards the future potential of artificial
Decision tree 0.156 0.395
intelligence, which has been proclaimed to trigger a digital trans­
Random forest 0.155 0.394 formation in customer experiences. Automation technologies are
Gradient boosted trees 0.154 0.393 already gradually being adopted as a means to overcome the current
Support vector machine 0.176 0.419 challenges related to social distancing and the changes in consumer
Note: the metrics range from 0 to ∞. behaviour (Seyitoğlu & Ivanov, 2020). Despite robots having the po­
tential to revitalise the service industries (Auliawan and Ong, 2020),
emotions are still inevitable elements in service encounters (Ngan and
Table 4 Yu, 2019; Woo and Chan, 2020). For that reason, this research in­
Local weights of emotional expressions. tertwines the lens of emotional contagion with human-robot
Attributes Weight interactions.
Surprise 0.156
On a more general level, this research first indicates that although
Happiness 0.088 the emotional robot, Sophia, can emulate a wide range of human facial
Neutral 0.066 expressions (Faraj et al., 2020), its images presented on Instagram fail to
Sadness 0.027 sufficiently demonstrate the feelings of fear, contempt, anger, and
Note: weights range from 0 to 1. disgust, making it impossible to train the machine learning models. In
fact, since Instagram allows users to reach a wide range of audiences,
In addition, this study provides the optimal combination of inputs these findings are not surprising and can be explained from the
required to maximise the forecast of the learning model. The technique perspective of impression management (Hong et al., 2020). Rooted in
is known as prescriptive analytics, aiming to uncover the best course of the discipline of psychology, impression management refers to the (sub)
action (Bertsimas and Kallus, 2020). In essence, it reverses the learning conscious processes in which people control how they are perceived by
procedure by defining a desired output first and refining input features others during social interaction (Hong et al., 2020). For example, in the
accordingly. To ensure that only sensible and feasible solutions are services industries, employees can purposely include or exclude certain
created, the constraints of the input features were defined to stay within traits so as to leave a lasting impression on consumers. Keeping this
2 standard deviations of the average value and to remain above/below notion in mind, while the pictures used for this study were not posted by
its minimum/maximum value. Based on the optimisation algorithm, Sophia itself, it could be the case that the robotics company prefers to
Fig. 1 visualises and predicts the optimal combination of Sophia’s share the “ideal” version of Sophia. This assumption can be proven
emotional expressions needed to reach the most positive sentiment further considering the dominated, and mostly positive, emotions pre­
sented by Sophia (e.g., happiness) on Instagram.

Fig. 1. Optimisation of machine learning model.

5
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

However, this study implies that Instagram users tend to use negative 2020a). The development of robots with emotional intelligence addi­
terms/phrases when leaving a comment on Sophia’s posts. These find­ tionally brings a major paradigm shift to how individuals interact with
ings are rather inconsistent with earlier studies in terms of online users’ robots. By learning from the most advanced human-like emotional
overall feelings towards robots. Even though several researchers claim robot, Sophia, this study crosses disciplinary boundaries as it connects
anthropomorphic robots to be more preferable (Christou et al., 2020; psychology and data science and provides cutting-edge perceptions of
Ivanov et al., 2018), a study conducted by Yu (2020b), investigating the robotics and artificial intelligence research in the context of service
humanoid robot at Henn-na hotel, suggested that robots with an experiences. As humans often personify computers/machines (McGinn
extremely high level of human-likeness can potentially lead to a sense of et al., 2018), this study uses a modern approach of applying social media
discomfort. Likewise, another recent study assessing hotel service robots (i.e., Instagram) data in order to advance the current understanding of
with different degrees of human-likeness concluded that users are less human-robot relationships. In addition, this research goes beyond the
likely to accept highly anthropomorphic robots due to safety concerns, traditional scope of a well-researched human social phenomenon by
the imitation of human behaviour, and the capabilities of robots to highlighting the possibility of emotional transmission, or emotional
handle multiple tasks (Jia et al., 2021). Furthermore, the explorative contagion, among any “organism”.
nature of this research poses a question of whether potential consumers Methodologically speaking, the explorative nature of this study un­
are able to make accurate inferences about how they would interact with covers observations (e.g., expression of surprise) that may have been
or react to an emotional robot in a real-life situation. As explained by more difficult to recognise otherwise. It leverages advanced analytics so
Grunder and Neuhofer (2021), insufficient interaction might evoke as to open up new avenues of research and practical applications.
hesitation for consumers since they are unsure of a robot’s capability to Resting on the concept of emotional contagion, the findings zoom in on
truly understand their feelings and emotions. the interplay between a robot’s emotional display and a consumer’s
When looking at the interplay of different emotions and Instagram potential reactions during service encounters in the near future.
users’ feelings, this research reveals an interesting result. In particular, Although this research is incapable of providing an exact answer to the
the findings suggest that the expression of surprise comes across as the question “when will human-level AI become a reality?” posed by Grunder
most influential emotion to the observers. Notably, surprise is a complex and Neuhofer (2021), this study calls for a need to further investigate
feeling and might carry any valence (i.e., neutral, negative, or positive) robots with emotions. As a value-inducing technology for co-creating
(Noordewier and Breugelmans, 2013). Hence, observers’ affective states service experiences, examining robotics within the contexts of con­
vary according to the nature of the surprising stimulus (Noordewier and sumer behaviour, design research, and psychological studies should be
Breugelmans, 2013). Albeit the fact that Sophia presents positive sur­ viewed as a relevant priority in academia.
prise, Noordewier and van Dijk (2019) indicate that observers may still
experience a state of interruption before they can truly reflect on their 6.2. Managerial implications
feeling towards the stimulus. Logically, due to the lack of social inter­
action (Grundner and Neuhofer, 2021) with Sophia, observers’ This research presents several implications for service businesses
emotional reactions might stay at the initial phase without thinking or that intend to employ robots with emotional intelligence as well as for
processing any further (Noordewier and van Dijk, 2019). Another robotics companies to optimise the design of human-like robots. First
plausible reason could be that since surprise is often related to a and foremost, the findings imply that the portrayal of the emotion sur­
freezing/suspension-like behaviour, viewers might present passive and prise requires supplementary adjustments. More specifically, as surprise
defensive reactions (Scherer et al., 2004) towards Sophia. can be interpreted as either positive or negative, it is critical to re-
From a human-likeness perspective, the results of this study could evaluate what kind of messages service robots intend to deliver. Tech­
also be explained in terms of the theory of uncanny valley (Appel et al., nicians and designers might need to adjust the intensity of the robot’s
2020; Mathur et al., 2020). Overall, even when Sophia portrays the facial muscles (Lucey et al., 2010) triggering surprise; namely, the inner
feeling of happiness, the polarity of the Instagram comments leans more and outer brow raisers, the upper lid raiser, the lips part, and the mouth
towards the negative side of the spectrum. Seeing that Sophia is the first stretch. In the same vein, it could be important for technicians to test the
anthropomorphic robot to display various emotions, it could be that optimal intensity in relation to the cheek raisers, the lip corner pullers,
online users are either not used to robots highly resembling humans (Yu, and the lips part for the expression of happiness (Lucey et al., 2010).
2020b) or that the effectiveness and authenticity of emotions presented Despite the fact that, based on the comments of the pictures, observers
by robots are still questionable (Wirtz et al., 2018). Furthermore, this did not present a very positive attitude towards Sophia, the results
study implies that the expression of sadness presents the lowest contri­ provide robotics companies with guidelines to optimise observers’
bution to the valence of comments. Returning to the topic of emotional emotional reactions. In particular, this study suggests that the combi­
regulation, since, assumingly, people are more likely to hinder and hide nation of the facial expressions happiness and surprise can reach the best
their negative feelings (Liu and Liu, 2020), the display of sadness may outcomes (see Fig. 1). On that note, the display of positive surprise is
generate a lower chance of transmission to Instagram users as a result. more likely to prevent viewers from having negative feelings towards
Finally, while this study places a particular focus on facial expres­ Sophia. In addition, this research also indicates that the activation of
sions and emotional reactions, emotions, per se, might not necessarily facial muscles for sadness (i.e., inner brow raiser, outer brow raiser,
lead to Instagram viewers’ positive or negative responses. However, this brow lowerer, lip corner depressor, and chin raiser) (Lucey et al., 2010)
study underlines the display of emotions as one of the subtle yet should be minimised so as to avoid any negative reactions and
compelling factors that could imaginably make a difference in the impressions.
emerging theme of human-anthropomorphic robot interaction.
6.3. Limitations and recommendations for future research
6. Conclusion
This study is not without its limitations. Primarily, it is critical to
6.1. Theoretical contribution note that although the service industries, including retail, hospitality,
and healthcare, amongst others, could benefit from emotional robots,
Among various nonverbal cues, emotions enable individuals to build Sophia has yet to be employed and implemented into real-life condi­
deeper and more meaningful relationships (Otterbring, 2017; Woo and tions. As such, it is also necessary to bear in mind that the Instagram
Chan, 2020). Alongside the emergence of robots, designing experiences commenters (referred to as potential consumers) in this study did not
that touch humans emotionally is of the utmost importance for re­ personally interact with any robots. Thus, since Sophia has not been
searchers and practitioners in any service environment (Neuhofer et al., used as a service robot, Instagram users may have a slightly different

6
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

mindset compared to consumers. Instead, the current practice seems to Christou, P., Simillidou, A., Stylianou, M.C., 2020. Tourists’ perceptions regarding the
use of anthropomorphic robots in tourism and hospitality. Int. J. Contemp. Hospit.
be much closer to previous research in which vignettes were applied in
Manag. 32 (11), 3665–3683.
order to examine human-robot interaction (e.g., Yu and Ngan, 2019). Costa, A., Guerreiro, J., Moro, S., Henriques, R., 2019. Unfolding the characteristics of
However, the more urgent and important message is to focus on envi­ incentivized online reviews. J. Retailing Consum. Serv. 47, 272–281.
sioning the power of emotional expression and encouraging researchers Decety, J., Ickes, W.J., 2011. The Social Neuroscience of Empathy. The MIT Press.
Ebrahimi, M., Mohammadi-Dehcheshmeh, M., Ebrahimie, E., Petrovski, K.R., 2019.
to be prepared for the emerging practices in future human-robot inter­ Comprehensive analysis of machine learning models for prediction of sub-clinical
action and communication. mastitis: deep Learning and Gradient-Boosted Trees outperform other models.
Meanwhile, as feelings based on Instagram comments do not fully Comput. Biol. Med. 114, 103456.
Elragal, A., Klischewski, R., 2017. Theory-driven or process-driven prediction?
represent the internal states of consumers, it is recommended to Epistemological challenges of big data analytics. Journal of Big Data 4 (1), 1–20.
consider experimental design and to incorporate other biometrics, such Fan, J., Yue, W., Wu, L., Zhang, F., Cai, H., Wang, X., Lu, X., Xiang, Y., 2018. Evaluation
as eye-tracking, electroencephalography, or galvanic skin response, in of SVM, ELM and four tree-based ensemble models for predicting daily reference
evapotranspiration using limited meteorological data in different climates of China.
future studies. Scholars could also apply other text analytics such as Agric. For. Meteorol. 263, 225–241.
topic modelling to disclose the content of comments together with the Faraj, Z., Selamet, M., Morales, C., Torres, P., Hossain, M., Lipson, H., 2020. Facially
sentiment of texts. Furthermore, facial recognition algorithms are often Expressive Humanoid Robotic Face. HardwareX, e00117.
Frommer, D., 2015. Toshiba’s New Humanoid Will Give Tokyo Mall Visitors Information
trained on exaggerated human facial expressions, questioning to what in Three Languages. https://qz.com/543076/toshibas-new-humanoid-will-give-to
extent machine learning can truly recognise emotions presented by kyo-mall-visitors-information-in-three-languages/.
human-like robots. Likewise, while this research assumed that big data Fuentes-Moraleda, L., Díaz-Pérez, P., Orea-Giner, A., Muñoz-Mazón, A., Villacé-
Molinero, T., 2020. Interaction between hotel service robots and humans: a hotel-
methods could be leveraged in this area, some of the common emotional
specific Service Robot Acceptance Model (sRAM). Jourism.Manag. Perspect. 36,
expressions were not viable due to the fact that these expressions ulti­ 100751.
mately did not contribute to the machine learning model. To address this Gendron, M., Crivelli, C., Barrett, L.F., 2018. Universality reconsidered: diversity in
issue, scholars are urged to involve human-raters to supplement and making meaning of facial expressions. Curr. Dir. Psychol. Sci. 27 (4), 211–219.
Gilbert, C.H.E., Hutto, E., 2014. Vader: a parsimonious rule-based model for sentiment
validate the machine learning categorisation results. Finally, one should analysis of social media text. Eighth Int.Conf. Weblogs.Media.Soc 81, 82–92.
note that the current study’s findings are only representative in the Gohary, A., Hamzelu, B., Pourazizi, L., Hanzaee, K.H., 2016. Understanding effects of co-
context of Instagram. Once the development of robots with emotional creation on cognitive, affective and behavioral evaluations in service recovery: an
ethnocultural analysis. J. Retailing Consum. Serv. 31, 182–198.
intelligence becomes more mature in the future, scholars are suggested Gong, H., Sun, Y., Huang, B., 2019. Gradient boosted models for enhancing fatigue
to continue this line of research by investigating whether emotional cracking prediction in mechanistic-empirical pavement design guide. J. Transport.
contagion still applies in human-robot interaction. Eng., Part B: Pavements 145 (2), 4019014.
Grundner, L., Neuhofer, B., 2021. The bright and dark sides of artificial intelligence: a
futures perspective on tourist destination experiences. J.Destin.Market.Manag 19,
Funding source 100511.
Hasegawa, N., Takahashi, Y., 2019. How recognition of human facial expression can be
incoporated in robot control. In: 2019 20th International Conference on Research
This study was supported by Ministry of Science and Technology, and Education in Mechatronics. IEEE, pp. 1–6.
Taiwan (Grant number: MOST 108-2218-E− 468-003). Hatfield, E., Bensman, L., Thornton, P.D., Rapson, R.L.(, 2014. New perspectives on
emotional contagion: a review of classic and recent research on facial mimicry and
contagion. Interpersona: Int. J.Pers. Relat. 8 (2), 159–179.
Acknowledgements Hong, S., Jahng, M.R., Lee, N., Wise, K.R., 2020. Do you filter who you are?: excessive
self-presentation, social cues, and user evaluations of Instagram selfies. Comput.
Hum. Behav. 104, 106159.
The authors would like to express their appreciation to Ms. Michelle
Io, H.N., Lee, C.B., 2020. Social media comments about hotel robots. J. China Tourism
Mattuzzi, a Research Master student in Language and Cognition at the Res. 16 (4), 606–625.
University of Groningen, for proofreading the manuscript. Michelle can Isen, A.M., 2009. A role for neuropsychology in understanding the facilitating influence
of positive affect on social behavior and cognitive processes. In: Lopez, S.J.,
be contacted via email on mattuzzi.michelle@gmail.com.
Snyder, C.R. (Eds.), Oxford Library of Psychology. Oxford Handbook of Positive
Psychology. Oxford University Press, pp. 503–518.
References Ivanov, S., Webster, C., Garenko, A., 2018. Young Russian adults’ attitudes towards the
potential use of robots in hotels. Technol. Soc. 55, 24–32.
Ivanov, S., Seyitoğlu, F., Markova, M., 2020. Hotel managers’ perceptions towards the
Appel, M., Izydorczyk, D., Weber, S., Mara, M., Lischetzke, T., 2020. The uncanny of
use of robots: a mixed-methods approach. Inf. Technol. Tourism 22 (4), 505–535.
mind in a machine: humanoid robots as tools, agents, and experiencers. Comput.
Jia, J.W., Chung, N., Hwang, J., 2021. Assessing the Hotel Service Robot Interaction on
Hum. Behav. 102, 274–286.
Tourists’ Behaviour: the Role of Anthropomorphism. Industrial Management & Data
Arora, A., Bansal, S., Kandpal, C., Aswani, R., Dwivedi, Y., 2019. Measuring social media
Systems ahead-of-print.
influencer index-insights from Facebook, Twitter and Instagram. J. Retailing
Joe, W.Y., Song, S.Y., 2019. Applying human-robot interaction technology in retail
Consum. Serv. 49, 86–101.
industries. Int. J. Mech.Eng. Robot Res. 839–844.
Atzeni, M., Recupero, D.R., 2020. Multi-domain sentiment analysis with mimicked and
Li, Y., Ishi, C.T., Ward, N., Inoue, K., Nakamura, S., Takanashi, K., Kawahara, T., 2017.
polarized word embeddings for human–robot interaction. Future Generat. Comput.
Emotion recognition by combining prosody and sentiment analysis for expressing
Syst. 110, 984–999.
reactive emotion by humanoid robot. In: 2017 Asia-Pacific Signal and Information
Auliawan, A.G., Ong, S., 2020. The usage of AI robot in English language teaching for
Processing Association Annual Summit and Conference. IEEE, pp. 1356–1359.
city revitalization case study: toda daini elementary school, toda city, saitama,
Liu, X.Y., Liu, Y., 2020. The service smile chain: linking leader emotions to customer
Japan. IOP Conf. Ser. Earth Environ. Sci. 436, 12022.
outcomes. Serv. Ind. J. 40 (5–6), 415–435.
Balazka, D., Rodighiero, D., 2020. Big data and the little big bang: an epistemological (R)
Liu, X.Y., Chi, N.W., Gremler, D.D., 2019. Emotion cycles in services: emotional
evolution. Frontiers in Big Data 3, 1–13.
contagion and emotional labor effects. J. Serv. Res. 22 (3), 285–300.
Bertsimas, D., Kallus, N., 2020. From predictive to prescriptive analytics. Manag. Sci. 66
Lu, L., Cai, R., Gursoy, D., 2019. Developing and validating a service robot integration
(3), 1025–1044.
willingness scale. Int. J. Hospit. Manag. 80, 36–51.
Biyik, C.A., Tanyeri, M., 2018. Pricing decisions in liner shipping industry: a study on
Lu, J., Su, X., Diao, Y., Wang, N., Zhou, B., 2021. Does online observational learning
artificial neutral networks. J. Market. Market. Res. 21, 125–150.
matter? Empirical evidence from panel data. J. Retailing Consum. Serv. 60, 102480.
Buzova, D., Sanz-Blas, S., Cervera-Taulet, A., 2019. Does culture affect sentiments
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I., 2010. The
expressed in cruise tours’ eWOM? Serv. Ind. J. 39 (2), 154–173.
extended cohn-kanade dataset (ck+): a complete dataset for action unit and
Celuch, K., 2020. Customers’ experience of purchasing event tickets: mining online
emotion-specified expression. In: 2010 IEEE Computer Society Conference on
reviews based on topic modeling and sentiment analysis. Int. J. Event .Festiv. Manag
Computer Vision and Pattern Recognition-Workshops. IEEE, pp. 94–101.
12 (1), 36–50.
Lundqvist, K., Liyanagunawardena, T., Starkey, L., 2020. Evaluation of student feedback
Chatterjee, A., Umang, G., Manoj, K.C., Radhakrishnan, S., Michel, G., Puneet, A., 2019.
within a MOOC using sentiment analysis and target groups. Int. Rev. Res. Open Dist.
Understanding emotions in text using deep learning and big data. Comput. Hum.
Learn. 21 (3), 140–156.
Behav. 93, 309–317.
Mathur, M.B., Reichling, D.B., Lunardini, F., Geminiani, A., Antonietti, A., Ruijten, P.A.,
Chen, X., Li, G., Yang, L., Nie, Q., Ye, X., Liang, Y., Xu, T., 2020. Profiling unmanned
Levitan, C.A., Nave, G., Manfredi, D., Bessette-Symons, B., Szuts, A., Aczel, B., 2020.
aerial vehicle photography tourists. Curr. Issues Tourism 23 (14), 1705–1710.
Uncanny but not confusing: multisite study of perceptual category confusion in the
Choi, Y., Choi, M., Oh, M., Kim, S., 2020. Service robots in hotels: understanding the
Uncanny Valley. Comput. Hum. Behav. 103, 21–30.
service quality perceptions of human-robot interaction. J. Hospit. Market. Manag. 29
(6), 613–635.

7
S.H.-W. Chuah and J. Yu Journal of Retailing and Consumer Services 61 (2021) 102551

McGinn, C., Cullinan, M.F., Culleton, M., Kelly, K., 2018. A human-oriented framework Rymarczyk, K., Żurawski, Ł., Jankowiak-Siuda, K., Szatkowska, I., 2019. Empathy in
for developing assistive service robots. Disability and Rehabilitation. Assist. Technol. facial mimicry of fear and disgust: simultaneous EMG-fMRI recordings during
13 (3), 293–304. observation of static and dynamic facial expressions. Front. Psychol. 10, 701.
Mende, M., Scott, M.L., van Doorn, J., Grewal, D., Shanks, I., 2019. Service robots rising: Ryo, M., Rillig, M.C., 2017. Statistically reinforced machine learning for nonlinear
how humanoid robots influence service experiences and elicit compensatory patterns and variable interactions. Ecosphere 8 (11), e01976.
consumer responses. J. Market. Res. 111 (10), 535–556. Scherer, K.R., Zentner, M.R., Stern, D., 2004. Beyond surprise: the puzzle of infants’
Mich, L., 2020. Artificial intelligence and machine learning. Handbook of E-Tourism expressive reactions to expectancy violation. Emotion 4 (4), 389–402.
1–21. Seyitoğlu, F., Ivanov, S., 2020. Service robots as a tool for physical distancing in tourism.
Murphy, J., Gretzel, U., Pesonen, J., 2019. Marketing robot services in hospitality and Curr. Issues Tourism 1–4.
tourism: the role of anthropomorphism. J. Trav. Tourism Market. 21 (1), 1–12. Shafqat, W., Byun, Y.C., 2020. A recommendation mechanism for under-emphasized
Neuhofer, B., Celuch, K., To, T.L., 2020a. Experience design and the dimensions of tourist spots using topic modeling and sentiment analysis. Sustainability 12 (1),
transformative festival experiences. Int. J. Contemp. Hospit. Manag. 32 (9), 1–26.
2881–2901. Shank, D.B., Graves, C., Gott, A., Gamez, P., Rodriguez, S., 2019. Feeling our way to
Neuhofer, B., Magnus, B., Celuch, K., 2020b. The impact of artificial intelligence on event machine minds: people’s emotions when perceiving mind in artificial intelligence.
experiences: a scenario technique approach. Electron. Mark. 1–17. Comput. Hum. Behav. 98, 256–266.
Ngan, H.F.B., Yu, C.E., 2019. To smile or not to smile – an eye-tracking study on service Söderlund, M., 2017. Employee display of burnout in the service encounter and its
recovery. Curr. Issues Tourism 22 (19), 2327–2332. impact on customer satisfaction. J. Retailing Consum. Serv. 37, 168–176.
Ngo, L.V., Nguyen, T.N.Q., Tran, N.T., Paramita, W., 2020. It takes two to tango: the role Stock, R.M., 2016. Emotion transfer from frontline social robots to human customers
of customer empathy and resources to improve the efficacy of frontline employee during service encounters: testing an artificial emotional contagion. Modell. Human
empathy. J. Retailing Consum. Serv. 56, 102141. Behavior in IS 1–12.
Nie, Z., 2020. Research on facial expression recognition of robot based on CNN Symons, J., Alvarado, R., 2016. Can we trust Big Data? Applying philosophy of science to
convolution neural network. In: 2020 IEEE International Conference on Power, software. Big Data & Society 3 (2), 2053951716664747.
Intelligent Computing and Systems. IEEE, pp. 1067–1070. Torre, I., Goslin, J., White, L., 2020. If your device could smile: people trust happy-
Nikou, M., Mansourfar, G., Bagherzadeh, J., 2019. Stock price prediction using DEEP sounding artificial agents more. Comput. Hum. Behav. 105, 106215.
learning algorithm and its comparison with machine learning algorithms. Intell. Tung, V.W.S., Au, N., 2018. Exploring customer experiences with robotics in hospitality.
Syst. Account. Finance Manag. 26 (4), 164–174. Int. J. Contemp. Hospit. Manag. 30 (7), 2680–2697.
Noordewier, M.K., Breugelmans, S.M., 2013. On the valence of surprise. Cognit. Emot. 27 Ustrov, Y., Valverde, M., Ryan, G., 2016. Insights into emotional contagion and its effects
(7), 1326–1334. at the hotel front desk. Int. J. Contemp. Hospit. Manag. 28 (10), 2285–2309.
Noordewier, M.K., van Dijk, E., 2019. Surprise: unfolding of facial expressions. Cognit. Willmott, C.J., 1982. Some comments on the evaluation of model performance. Bull. Am.
Emot. 33 (5), 915–930. Meteorol. Soc. 63 (11), 1309–1313.
Ogutu, J.O., Piepho, H.P., Schulz-Streeck, T., 2011. A comparison of random forests, Wirtz, J., Patterson, P., Kunz, W., Gruber, T., Lu, V.N., Paluch, S., Martins, A., 2018.
boosting and support vector machines for genomic selection. BMC Proc. 5 (3), 1–5. Brave new world: service robots in the frontline. J. Serv. Manag. 29 (5), 907–931.
BioMed Central. Woo, K., Chan, B., 2020. “Service with a smile” and emotional contagion: a replication
Otterbring, T., 2017. Smile for a while: the effect of employee-displayed smiling on and extension study. Ann. Tourism Res. 80, 102850.
customer affect and satisfaction. J. Serv. Manag. 28 (2), 284–304. Yan, Q., Zhou, S., Wu, S., 2018. The influences of tourists’ emotions on the selection of
Palagi, E., Nicotra, V., Cordoni, G., 2015. Rapid mimicry and emotional contagion in electronic word of mouth platforms. Tourism Manag. 66, 348–363.
domestic dogs. Royal Society Open Science 2 (12), 150505. Yu, C.E., 2018. Humanlike robot and human staff in service: age and gender differences
Pantano, E., 2020. Non-verbal evaluation of retail service encounters through in perceiving smiling behaviors. In: 2018 7th International Conference on Industrial
consumers’ facial expressions. Comput. Hum. Behav. 106448. Technology and Management. IEEE, pp. 99–103.
Prentice, C., 2016. Leveraging employee emotional intelligence in casino profitability. Yu, C.E., 2020a. Emotional contagion in human-robot interaction. E-Review of Tourism
J. Retailing Consum. Serv. 33, 127–134. Research 17 (5), 793–798.
Prentice, C., Nguyen, M., 2020. Engaging and retaining customers with AI and employee Yu, C.E., 2020b. Humanlike robots as employees in the hotel industry: thematic content
service. J. Retailing Consum. Serv. 56, 102186. analysis of online reviews. J. Hospit. Market. Manag. 29 (1), 22–38.
Raffard, S., Bortolon, C., Khoramshahi, M., Salesse, R.N., Burca, M., Marin, L., Bardy, B. Yu, L., Liu, H., 2003. Feature selection for high-dimensional data: a fast correlation-based
G., Billard, A., Macioce, V., Capdevielle, D., 2016. Humanoid robots versus humans: filter solution. In: Proceedings of the 20th International Conference on Machine
how is emotional valence of facial expressions recognized by individuals with Learning, pp. 856–863.
schizophrenia? An exploratory study. Schizophr. Res. 176 (2–3), 506–513. Yu, C.E., Ngan, H.F.B., 2019. The power of head tilts: gender and cultural differences of
Recknagel, F., 2001. Applications of machine learning to ecological modelling. Ecol. perceived human vs human-like robot smile in service. Tour. Rev. 74 (3), 428–442.
Model. 146 (1–3), 303–310. Yu, C.E., Xie, S.Y., Wen, J., 2020. Coloring the destination: the role of color psychology
Ribeiro, M.T., Singh, S., Guestrin, C., 2016. Why should I trust you?" Explaining the on Instagram. Tourism Manag. 80, 104110.
predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Zeng, Z., Chen, P.J., Lew, A.A., 2020. From high-touch to high-tech: COVID-19 drives
Conference on Knowledge Discovery and Data Mining. ACM Press, pp. 1135–1144. robotics adoption. Tourism Geogr. 22 (3), 724–734.
Rincon, J.A., Costa, A., Novais, P., Julian, V., Carrascosa, C., 2019. A new emotional Zhou, H., Huang, M., Zhang, T., Zhu, X., Liu, B., 2017. Emotional Chatting Machine:
robot assistant that facilitates human interaction and persuasion. Knowl. Inf. Syst. 60 Emotional Conversation Generation with Internal and External Memory. ArXiv
(1), 363–383. Preprint ArXiv, 1704.01074.
Robotics, Hanson, 2020. Sophia. https://www.hansonrobotics.com/sophia/.