Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.

11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

LOCALIZING USER EXPERIENCE FOR MOBILE APPLICATION:

A CASE STUDY AMONG USM UNDERGRADUATES

Jap Show Fong1, Lok Li Ching2, Khor Choon Chiat3, Nurul Hashimah
Ahamed Hassain Malim4, Mohd Heikal Husin5 and Manmeet Mahinderjit
Singh6
1
Universiti Sains Malaysia, Malaysia, jsfoong.ucom12@student.usm.my
2
Universiti Sains Malaysia, Malaysia, llching.ucom12@student.usm.my
3
Universiti Sains Malaysia, Malaysia, kcchiat.ucom12@student.usm.my
4
Universiti Sains Malaysia, Malaysia, nurulhashimah@usm.my
5
Universiti Sains Malaysia, Malaysia, heikal@usm.my
6
Universiti Sains Malaysia, Malaysia, manmeet@usm.my

ABSTRACT. The increasing number of mobile devices consumptions, es-

pecially mobile phones and smartphones had caused a growing interest in
the user experience research on the mobile platform. However, it is difficult
to gain an encompassing understanding of the user experience especially
from a localized context. This paper presents our survey finding on the un-
derstanding of user experience among USM undergraduates. The research
finding identifies that a respondents’ field of study has some influence on
their understanding of user experience. Overall, the respondents tend to
agree that user experience is subjective and based on the individual’s inter-
action with an application. The ISO definition of user experience is dis-
cussed and compared with the survey results. The comparison highlights
that the definition of user experience is similar with the ISO’s definition of
user experience.

Keywords: user experience, user satisfaction, application performance, ap-

plication design

INTRODUCTION
The International Telecommunication Union estimates that there are nearly 7 billion mo-
bile subscriptions worldwide by May 2014. This estimate is equivalent to 95.5 percent of the
world population (MobiForge, 2014). The statistic indicates that almost everyone in the world
uses a mobile device that indirectly triggers an increased in the development of mobile appli-
cations. This increase could evidently be seen in the exponential growth of the mobile appli-
cation development since the launch of the Apple app store in July 2008 (MobiForge, 2014).
The variety of choices of mobile applications makes the mobile application market competi-
tive. Thus, the user satisfaction and experience has become a crucial aspect in determining the
success of mobile application (Chen & Zhu, 2011). User satisfaction is referred to as the level
of happiness of users while using any products or services (Law et al., 2009). The user satis-
faction indirectly reflects the quality of the products or services provided to the users, and it is
generally difficult to measure. This difficulty is due to user satisfaction depends on the emo-
tional level of the users while using the product or service (Swallow, Blythe & Wright, 2005).
There are external factors that influences the emotion of the users before using the products or

366
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

services, and subsequently causes the users to feel satisfied or dissatisfied with the their us-
age. The interest of users while using mobile applications is imperative from a sustainability
perspective. Therefore, the users’ satisfaction and experience with the mobile application
needs to be measured effectively so the software developer’s effort will not be in vain. Many
researchers have examined the user satisfaction and experience on mobile application such as
Chen and Zhu (2011), Law et al. (2009), Swallow, Blythe and Wright (2005), Osman et al.
(2012), Hu and Liu (2010) as well as Arhippainen and Tahti (2003). In the study by Hu and
Liu (Osman et al., 2012), they suggested that the ‘user experience’ could impact the users’
perception towards mobile games application. Their findings have been the underlying factor
for mobile games developers to consider on a more appealing mobile game design for the
users.
The work of Law et. al. (2009) provided the definition of user experience via a survey
covering an enormous spectrum of users. These users are researchers and practitioners from
academia and industry. Their finding indicates that ‘user experience’ is dynamic, context-
dependent and subjective. Nevertheless, the ISO definition was found to be generally
acceptable by their respondents. They also recommended that the “term user experience could
be scoped to products, systems, and objects that a person interacts through as user interface”.
It was also highlighted that their definition is mainly influenced by socio-cultural factors such
as countries of origin while years of experience and workplace does not play a major
influence. In Malaysia, there are no existing studies, which focus on the interpretation of lo-
cals on the term ‘user experience’ in the context of mobile application. The closest existing
study that is related to user experience is a study on smartphone usage by Osman et al. (2012).
In their work, it was obvious that most of the smartphone users (60%) are within the age
range of 17 – 26 and most of them are undertaking their bachelor degrees (Hu & Liu, 2010).
Hence, this paper is aimed at understanding the factors that influence the user satisfaction and
experience on mobile application from a local context (Malaysia). The investigation further
focuses on Malaysian youths that reside within the specific age group that highly utilizes with
smartphones. By identifying the factors that affects user experience on mobile applications,
we hoped that this paper would help mobile application developers to have a clearer
perspective on specific criterias or requirements that would enhance their mobile application
experience.

BACKGROUND STUDY
The emergence of mobile applications has become vital in supporting human needs for in-
formation, communication or leisure (Ickin et al., 2012). The quality of user experience in
such applications depends on factors such as application interface design, application
performance, battery efficiency, phone features, application and data connectivity cost, user
routine and user lifestyle (Ickin et al., 2012). These factors was determined after an in-depth
investigation on various mobile applications such as Skype, Facebook, eBay, Worldwar and
more.
Lim (2013) stated that user experience should be measured based on users’ emotional re-
sponse in terms of the intrinsic, semantic and symbolic attractiveness of a product. Similar
observations was made by Chen and Zu (2011) where they highlighted that user experience is
related to personal feelings that is subjective and complex. Hence, an accurate assessment of
user satisfaction or user experience is made more complicated. Chen and Zu (2011) also
hypothesized that the user experience of a user on mobile applications are affected by the user
themselves, the software, and their context of use. They then proposed a four-dimension as-
sessment system to capture user experience in an accurate manner. These dimensions are user
characteristics, application properties, application system support and context parameters.

367
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

Based on Chen and Zu’s (2011) case study of a music application reveals that the most influ-
enced dimensions are the application properties and system support. This results are based on
their finding that indicates that the overall quality of the application and its properties affect
user experience specifically, on the level of quality and acoustic fidelity for the users. The
speed of the mobile network and the phone’s operating system also seem to be a high concern
among the users as well.
Another work by Toshihiko Yamakami (2012) highlighted four factors that affect user
satisfaction such as emotion, simplicity, ecosystem in the service and, the iterated
improvement of the system. The emotion factor is a strong driving factor for human behavior
hence they proposed that the mobile application design should address emotion engineering.
On the other hand, emotion simplicity refers to the dissatisfaction felt by the user that led
them to stop using the application. The ecosystem in the service focuses on facilitating users
in creating their personalized satisfaction system. The last factor highlights the importance of
the mobile application to be continuously improved according to the evolving users’ require-
ments.

METHODOLOGY
The target group being focused in this study are university students undertaking their
bachelor degrees within the age range of 19 – 25 years old. We decided to utilize our local
study environment (Universiti Sains Malaysia) as a case study and distributed our survey
online via Google Docs due to the widespread of the students within the campus. The ques-
tionnaire was made available to the respondents via a URL link. We promoted the online sur-
vey to USM students by posting the link on several Facebook groups that are made for the
USM community, including USM Info Sharing Corner and USM Computer Science Intake
2012/2013. Besides that, personalized invitations to some USM students to answer the ques-
tionnaire was also made by sending them the link for the questionnaire using the messaging
service on Facebook. The response duration for the questionnaire was limited to two weeks.

RESULTS AND DISCUSSION

Respondents’ background
A total of 94 respondents from various course backgrounds responded to the online sur-
vey. Table 1 shows the general profile of the respondents.
Table 1. General profiles of the respondents
Variable Frequency Percentage (%)
My field of study/ expertise is …
Computer Science 41 43.62
Art 11 11.70
Applied Sciences and Engineering 9 9.58
Other 33 35.11
My understanding with the term "user experience"…
Fully understand 11 11.70
Mostly 49 52.13
Little 27 28.72
Not at all 7 7.45
My experience with mobile phones
1 year 3 3.33
2 years 7 7.78
3 years 23 25.56
More than 3 years 61 63.33

368
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

There is a variation in the number of years of experience with mobile phones. Many of the
respondents reported they had 3 years of experience with mobile phones (mode = 3 years)
while most of the respondents had more than 3 years of experience with mobile phones
(64.89%). The mean for number of years of experience with mobile phones is 5.88 years
while the standard deviation is 16.49. The mean signifies a wide variation in the number of
years of experience among the respondents. Among the 61 respondents who reported having
more than three years of experience with mobile phones, 12 of them reported having ten years
or more experience with mobile phones. Only ten respondents (10.64%) had less than three
years’ experience with mobile phones where the number of respondents with 1-year experi-
ence and two years’ experience are 3 and 7 respectively.
From the survey, 11 respondents (11.70%) indicated that they have a full understanding of
the term ‘user experience’; whereas another 49 respondents reported to understand mostly the
meaning of the word. The result shows that most of the respondents had a good understanding
of the concept of ‘user experience’. However, there were seven respondents who claimed they
do not understand the term ‘user experience’ at all. We are interested to find out if there is any
relation between the users’ field of study and their understanding of the term ‘user experi-
ence’.
Table 2. The level of expertise and understanding of ‘user experience’
Category Fully Understand (%) Mostly (%) Little (%) Not at all (%)

Art 27.27 18.18 36.36 18.18

Applied Science 0 44.44 44.44 11.11

Computer Science 12.20 70.73 14.63 2.44

Others 9.09 42.42 39.39 9.09

Based on Table 2, respondents with the field of study in Computer Science had better un-
derstanding of the term ‘user experience’. This result is indicated by lower percentage of re-
spondents who had no knowledge in ‘user experience’ (2.44%) and a high percentage of re-
spondents that fully and mostly (82.93%) understand what ‘user experience’ means. The high
percentage might be because Computer Science student are more exposed to the concept of
‘user experience’ via other applications such as web based, standalone and mobile applica-
tion. It is also interesting to note that nearly 30% from the Arts field claimed that they fully
understand the term. For other categories, there was no significant difference between re-
spondents who had good understanding (fully understand and mostly understand) and poor
understanding on ‘user experience’ (little and no knowledge in user experience).
User experience results
Table 3 shows 22 statements about ‘user experience’ that were asked in the survey. Re-
spondents were required to indicate their level of agreement with these statements with a five-
point Likert scale. The five point Likert scale for the statements was used to show the
respondents’ level of agreement with the statement, “1” indicates strongly disagree, whilst
“5” indicates strongly agree, and “3” indicates neutral. These 22 statements can be catego-
rized into two main categories, general statements about ‘user experience’ on mobile plat-
forms and the factors affecting user experience on mobile platforms. The second category is
further divided into three factors: user characteristics, context of using mobile applications
and product (mobile application) characteristics. The response rate for all the statements were
100%. However, the response of the 7 respondents, who reported having no understanding on
the term ‘user experience’, were excluded from the analysis. There are six statements with
mean value higher than 4, which were statement 1, 6, 10, 13, 15 and 17, with the highest be-

369
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

ing statement 10 (mean = 4.15). The top two statements with the highest level of agreement
indicated there was a common agreement among the respondents that ‘user experience’ is
important for applications. Respondents tend to agree with most of the statements on user
experience. Out of the 22 statements on user experience, there were only six statements
(27.27%) that obtained at least one “strongly disagree” (point scale = 1) from the respondents.
Generally, all the statements receive favourable agreements from the respondents. No wide
variation was discovered among the three factors affecting user experience, and the respond-
ents had a similar level of agreement on these three factors though the level of agreement for
each statement varies. There are no definite indication which factors was strongly agreed by
the respondents.
In general, all respondents agreed that user experience is something subjective and related
to the personal feeling. The level of agreement among the respondents on the 22 user experi-
ence-related statements were quite similar, and no significant variation were found. Sixteen
out of the 22 statements (72.73%) had mean agreement levels somewhere between 3.5 (mid-
point between neutral and agree) and 4 (agree). The result indicates that the respondents tend
to agree with most of the statements. There was only six statements (22.27%) that obtained
mean level of agreements higher than four on the 5-point scale. However, the mean level of
agreements of these 6 statements were between 4.05 and 4.15, indicating respondents agree
with those statements, but their level of agreements are not strong (5 = strongly agree), and
could be considered as mild agreement. The respondents’ agreement levels on three factors
affecting user experience (context, user factor and product characteristic) were roughly the
same, between 3.5 and 4.0.
Respondents’ definition of User Experience
The respondents were required to give the definition of user experience and the responds
obtained were analyzed to capture what user experience meant for the respondents. Keywords
from the users’ definitions were analyzed and group into four different categories to under-
stand what the user experience meant for the respondents. The four categories are “What is
user experience”, “What constitutes a good user experience? (Factors)”, “Key ideas about
user experience” and “What to measure in evaluating user experience?”. Among the 94 re-
spondents who responded to this questionnaire, 13 respondents (13.83%) did not give any or
relevant definitions about user experience.

Table 3. Statements on User Experience by category

Category # Statement M SD
General 1 User experience determines the success of an application. 4.06 0.57
Statement 2 User experience concerns the qualitative value of an 3.89 0.48
3 application. 3.75 0.40
The design for user experience should be a qualitative
4 approach rather than quantitative approach. 3.70 0.38
5 It is hard to define what user experience is but we may feel 3.74 0.40
6 it. 4.14 0.61
User experience is dynamic and flexible. It is not rigid.
7 Application with better user experience would eventually 3.92 0.49
gain a larger amount of users.
8 Perceived enjoyment of mobile application users, when 3.46 0.25
using and after using mobile applications, is a good way to
9 estimate the user experience. 4.15 0.62
10 Perceived enjoyment of mobile application users could not 3.80 0.43
be measured quantitatively.
11 Understanding the users is crucial for the design of user 3.61 0.33
experience

370
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

User experience is the most important aspect of an

application, besides its functionality.
User experience covers every aspect of an application or
system.
Factor: 12 Prior experience with other similar applications would 3.86 0.46
User factor affect the user experience
13 People with different backgrounds would have different 4.07 0.57
user experience on the same application.
Factor: 14 User experience is something context dependent. (Hints: the 3.51 0.27
Context 15 environment) 4.05 0.56
The characteristics of the mobile hardware (screen size,
weight, dimension of the device, etc.) used for the mobile
application would also affect the users’ experience.
Factor: 16 The reputation of an application by other users would have 3.78 0.42
Product an impact on the user experience of other users.
characteristi 17 User experience is related by the ease of use, performance 4.07 0.57
c and responsiveness of a system.
18 Aesthetic characteristics of an application are crucial to the 3.69 0.37
creation of good user experience and satisfaction.
19 A good user interface is one of the main factors in creating 3.97 0.52
a great user experience.
20 In mobile applications with online functionality, the speed 3.92 0.49
of the Internet connection would have significant effect on
21 users’ satisfaction. 3.79 0.42
22 The ability to customize an application would improve user 3.97 0.52
satisfaction.
Mobile application should be improved and updated
regularly to meet the frequent changes in the user
requirements and environment factors.

Answers such as “I don’t know”, “not sure”, and more were excluded from our analysis to
ensure the accuracy of the collected data. The keywords from the respondents’ definitions
were identified and categorized into one of the four categories. Repeated keywords in differ-
ent respondents’ definitions were not shown in the table. Definitions with the same meaning,
for example “Easy to use” and “ease of use”, were grouped together. According to the
analysis as shown in Table 4, the respondents thought that user experience was something
personal and concerned personal interaction with the applications. For the respondents, fac-
tors affecting user experience in mobile applications were ease of use, functionality and per-
formance of the application and aesthetic value of the user interface. From the definition giv-
en by the respondents, user experience involves emotions of the users towards an application
making it a qualitative measure.

Table 4. Definition on user experience

What is user What constitutes a good Key ideas about What to measure in
experience user experience (factors) user experience evaluating user experi-
ence
Users’ feelings Ease of use Personalize Behaviours
User based Performance UI Design Attitude
User interaction responsiveness Creative Emotions
Subjective Creativity environment Happiness cum
Personal opinion Comfortable to use satisfaction
Safe Users’ preferences
Pleasant to use
Convenient

371
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

Easy to learn
Speed
System fluency
Ease of operation
Nice interface
Pleasant quality
functionality
Easy handling of
application

ISO definition on User Experience

It is interesting to compare the respondents’ definitions on user experience with the defini-
tion by the International Organization for Standardization. The international standard on
ergonomics of human system interaction defines user experience as "a person's perceptions
and responses that result from the use or anticipated use of a product, system or service"
(ISO, 2008). The definition of users experience by ISO is in line with the definitions given by
the respondents in terms of the subjectivity of the definition. The respondents agreed that user
experience was something personal, and it is the result of an individual’s interaction with
applications. Besides, the ISO definition and the respondents’ definition were also in line
about what user experience is. User experience concerns what the users feel as the result from
their use of an application. It is qualitative in nature, and varies from person to person. An
interesting observation is, the ISO definition included the anticipated use of a product, system
or service as part of what constitutes a user experience. However, the definitions given by
respondents did not mention any anticipated use of an application. For the respondents, the
anticipated use of an application is not a central part of user experience. Note that the ISO
definition on user experience applies to product, system or service while the context of this
survey is focusing on user experience on mobile application. The ISO definition focuses on a
much broader scope while the scope of user experience evaluated in this survey is smaller.

CONCLUSION
This survey paper aimed to discover the understanding of user experience among the Ma-
laysian youth, specifically the undergraduates, as a case study. In summary, we were able to
discover what user experience means to our youth, and how the target groups of respondents
view user experience. Their level of understanding on the term is very much related to their
field of studies. Respondents with field of study/expertise in Computer Science had a higher
level of understanding in user experience. Most respondents had viewed user experience as
something subjective, and highly personal. This is similar to the ISO definition on user expe-
rience. Besides, respondents also tend to agree with the three factors influencing user experi-
ence i.e. user factor, context and product characteristics. Further research should look into the
understanding of user experience among industrial practitioners and its relation with the field
of User Centered Design (UCD).

REFERENCES
MobiForge (2014). Global mobile statistics 2014 Part A: Mobile subscribers; handset market share;
mobile operators. (Research & Analysis). Retrieved from MobieForge website:
http://mobiforge.com/research-analysis/global-mobile-statistics-2014-part-a-mobile-subscribers-
handset-market-share-mobile-operators

Chen, Z, & Zhu, S. (2011). The Research of Mobile Application User Experience and Assessment
Model. International Conference on Computer Science and Network Technology (ICCSNT).
2832 – 2835.

372
 Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 Paper No.
11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) 206

Law, EL-C, Roto, V, Hassenzahl, M, Vermeeren, A.P.O.S, & Kort, J. (2009). Understanding, Scoping
and Defining User eXperience: A Survey Approach. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, 719-728.

Swallow, D, Blythe, M & Wright, P. (2005). Grounding Experience: Relating Theory and Method to
Evaluate the User Experience of Smartphones. Proceedings of the 2005 annual conference on
European association of cognitive ergonomics, 91 – 98.

Osman, M.A, Talib, A.Z, Sanusi, Z.A, Tan, SY & Alwi, A.S. (2012). A study of the Trend of
Smartphone and its Usage Behavior in Malaysia. International Journal on New Computer Archi-
tectures and Their Applications, 2 (1), 275 – 286.

Hu, F & Liu, Y. (2010). Impact of Experience and Gender Differences on Users’ Perceptions on Mo-
bile Game. International Conference on Multimedia Technology (ICMT).

Ickin, S., Wac, K, Fiedler, M, Janowski, L, Hong, JH & Dey, A.K. (2012). Factors Influencing Quality
of Experience of Commonly Used Mobile Applications. IEEE Communications Magazine, 4 (5).

Lim T.Y. (2013). User Experience Evaluation Methods for Mobile Devices. 2013 Third International
Conference on Innovative Computing Technology (INTECH). 281 – 286.

Yamakami, T. (2012). From User Experience to Social Experience: A New Perspective for Mobile
Social Game Design. 2012 9th International Conference on Ubiquitous Intelligence & Compu-
ting and 9th International Conference on Autonomic & Trusted Computing (UIC/ATC). 792 –
796.

Arhippainen, L. & Tähti, M. (2003). Empirical Evaluation of User Experience in Two Adaptive Mobile
Application Prototypes. Proceedings of the 2nd International Conference on Mobile ldots. 27 –
34.

ISO DIS 9241-210 (2008). Ergonomics of human system interaction - Part 210: Human-centred design
for interactive systems (formerly known as 13407). International Organization for Standardiza-
tion (ISO). Switzerland.

373