Challenges Types Emulators
Testing of Mobile Applications
Meenakshi D’Souza
International Institute of Information Technology Bangalore.
Challenges Types Emulators
Overview
Introduction to mobile apps testing.
Key challenges to address.
Types of mobile apps testing.
Some emulators and tools.
Challenges Types Emulators
Mobile apps testing
Mobile application testing is a process by which application
software developed for handheld mobile devices is tested for
its functionality, usability and consistency.
Can be automated or manual.
Mobile applications either come pre-installed or can be
installed from mobile software distribution platforms.
Related area: Testing of wearable applications.
Challenges Types Emulators
Key challenges for mobile applications testing
Device fragmentation: Wide range of mobile
platforms/operating systems.
Each operating system has its own limitations.
Each application has to be testing across multiple devices
running on the same platform. This has to be repeated for
several platforms.
Wide range of mobile devices with different screen sizes and
hardware configurations.
Gaining access to various devices for testing is expensive and
time consuming.
Challenges Types Emulators
Key challenges for mobile applications testing
Large number of mobile network operators.
It is estimated that there are over 400 mobile network
operators in the world.
Some of them are common standards like GSM and CDMA, a
large number of them are on less common standards.
Each network operator uses a different kind of network
infrastructure, limiting the flow of information.
Challenges Types Emulators
Key challenges for mobile applications testing
Executing a test script across devices is a challenge.
Devices differ in key strokes, input methods, texture, display
etc. Single script will not function on every device.
Mobile apps have to be tested for downloadability from a
particular store, across networks and devices.
Compatibility across devices, platforms and networks is
another key feature for testing.
Challenges Types Emulators
Mobile application vs. desktop application testing
On desktop, the application is tested on a CPU. On a mobile
device, the application is tested on mobile handsets.
Mobile device screen size is smaller than desktop.
Mobile devices have less memory than desktop.
Mobiles use network connections like 2G, 3G, 4G or WIFI
whereas desktops use broadband or dial up connections.
The automation tool used for desktop application testing
might not work on mobile applications.
Challenges Types Emulators
Types of mobile applications testing
To address the new challenges in mobile apps testing, there are
several different types of mobile apps testing, some of them just
tuned for mobile apps.
Functional testing: Functional testing ensures that the
application is working as per the requirements. Most of the
tests conducted for this are driven by the user interface and
call flow.
Usability testing: Testing done to make sure that the mobile
app is easy to use and provides a satisfactory user experience
to customers.
Key to an app’s commercial success.
Tests for uniform user experience across devices; addresses the
device fragmentation issues.
Tests for functionality, placement/sizing across different
devices.
Challenges Types Emulators
Types of mobile applications testing
Performace testing: Testing process done to check the
performance and behavior of the application under certain
conditions including
low battery, bad network coverage, low available memory,
simultaneous access to applications server by several users etc.
Performance testing is done on the client side and on the
server side.
Compatibility testing: Testing of the application in different
mobiles devices, browsers, screen sizes and OS versions etc.,
according to the requirements.
Includes testing for finer aspects like screen resolution, local
language features etc.
Challenges Types Emulators
Types of mobile applications testing
Memory-related testing: This testing is done to ensure that
the application manages memory and doesn’t slow down the
overall system.
Mobile devices have significant constraints on memory.
Applications should delete temporary files, use screens and
other hardware resources optimally (wakelocks) etc.
Installation testing:
Some applications come pre-installed on a devices, others have
to be downloaded and installed.
This testing covers installation, updating and uninstalling of an
application.
Challenges Types Emulators
Types of mobile applications testing
Interrupt testing: An application may face interrupts while
installing or while in use.
Interrupts include
Incoming and outgoing SMS and MMS, incoming and
outgoing calls, incoming notifications, cable insertion and
removal for data transfer, network outage and recovery, media
Player on/off, device power cycle etc.
Interrupt testing tests if an application correctly goes into a
suspended state and resumes afterwards.
Challenges Types Emulators
Types of mobile applications testing
Security testing: Several mobile phones are used for secure
transactions including bank transactions, credit card payments
etc.
Security testing checks for vulnerabilities involving hacking,
access control and authentication, data security, session
management etc.
There are several security standards like Common Criteria
(CC), NIST standard etc. that need to be adhered to.
This is a very involved area, not covering any depth here.
Different kinds of attacks: Hardware, network, application
software.
Program analysis, coding standards etc. play an important
role.
Extensive cryptography.
Challenges Types Emulators
Types of mobile applications testing
Location testing: Typically, connectivity changes with network
and location. Functionality of an application needs to be
tested for these changes.
Very difficult to mimic these changes in a
laboratory/development center and test.
Challenges Types Emulators
Types of mobile applications testing
Apart from the listed system level features, normal unit
testing, integration testing etc., are performed for mobile
phone apps.
These can be done using the criteria that we learnt in the
course and using mutation testing.
The listed system level testing are maunally done, cannot be
fully automated.
Challenges Types Emulators
Types of mobile applications testing
Summary slide on mobile application testing framework. Image
credit: Software Testing class website.
Challenges Types Emulators
Use of emulators in testing of mobile apps
Typical system level features that are tested for mobile apps
are tested using emulators.
Emulator is a system/software that runs a software from one
environment (platform) in another environment without
changing the software itself.
Emulators work by duplicating the features of a real system in
a software+system set-up.
Challenges Types Emulators
Types of mobile emulators
Device Emulator: Provided by device manufacturers.
Browser Emulator: Simulates mobile browser environments.
Operating systems Emulator: Provided by the OS firm (Apple
provides emulators for iOS, Microsoft for Windows phones
and Google for Android phones).
Challenges Types Emulators
Some (free) mobile phone emulators
Mobile Phone Emulator:
Used to test the display of a particular website on a cell phone
terminal.
Available for use from
http://www.mobilephoneemulator.com/.
mobiReady TM :
A free tool for testing website performance on mobile devices.
Available for use from https://ready.mobi/.
Challenges Types Emulators
Some other mobile apps testing tools
Ranorex ®:
A GUI testing tool for mobile apps.
Available for free trial from https://www.ranorex.com/.
Jamo Automator.
A scriptless cloud-based test automation tool for mobile apps
testing.
Free trail available from
https://www.jamosolutions.com/automator-1.
Challenges Types Emulators
Mobile apps testing: Summary
This lecture only gives an overview of mobile apps testing.
Main focus is on system-level testing of specific quality
attributes.
Needs extensive tool support, specific domain knowledge and
training.
