LECTURE 9

TESTING MOBILE APPLICATIONS

Dr. Asmau Usman, Nile University of Nig.

 Outline

➢ Testing Mobile Applications

➢ The strategies for testing Mobile App
➢ Considering the spectrum of user
interaction

2
Dr. Asmau Usman, Nile University of Nig.
Introduction

➢ The same sense of urgency that

drives WebApp projects also
pervades MobileApp projects.

➢ Stakeholders are worried that they will

miss a market window and press to
get the MobileApp into the store.

➢ Technical activities that often occur

late in the process, such as
performance and security testing, are
sometimes given short shrift.

10
Dr. Asmau Usman, Nile University of Nig.
 Cont.,

➢ Usability testing that should occur

during the design phase may end
up being deferred until just before
delivery.
➢ These can be a catastrophic
mistakes.
➢ To avoid this situation, you and
other team members must ensure
that each work product exhibits
high quality.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Mobile technology is simply ramping up

faster than other technologies in the past—
➢ it may be the fastest adoption curve in
history.
➢ And this has important implications for your
business model.
➢ You’ve got to be fast to market but also
prepared for rapid adoption.
➢ If your application performs poorly, or fails
under load, many competitors are ready to
take your place—the barriers to entry are
low.

11
Dr. Asmau Usman, Nile University of Nig.
Cont.,

➢ MobileApp requirements and design

models cannot be tested solely with
executable test cases.
➢ You and your team should conduct
technical reviews and test usability as well
as performance.
➢ Configuration testing is especially
important as a mechanism for verifying the
MobileApp’s ability to take context into
account.
➢ The intent is to uncover and correct errors
before the MobileApp is released to the
end-user community.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ There are several important questions to

ask when creating a MobileApp testing
strategy.
➢ Do you have to build a fully functional
prototype before you test with users?
➢ Should you test with the user’s real device
or provide a device (emulator) for testing?
➢ What devices and user groups should you
include in testing?
➢ What are the benefits/drawbacks of lab
testing versus remote testing?

Dr. Asmau Usman, Nile University of Nig.

 Mobile App Testing?

What is it? Who does it?

➢ MobileApp testing is a ➢ Software engineers and other
collection of related project stakeholders (managers,
activities with a single customers, end users) all
goal: participate in MobileApp testing.
➢ to uncover errors in Why is it important?
MobileApp content, ➢ If end users encounter errors or
function, usability, difficulties within the MobileApp,
navigability, they will go elsewhere for the
performance, capacity, personalized content and
and security. function they need.
➢ To accomplish this, a ➢ For this reason, you must work to
testing strategy that find and correct as many errors
encompasses both as possible before the
reviews and executable MobileApp is placed in an app
testing is applied. store or repository.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,
What are the steps?
➢ The MobileApp testing process begins by
focusing on user-visible aspects of the
MobileApp and proceeds to testing of
technology and infrastructure.
➢ Several testing steps are performed:-
➢ Content Testing
➢ Interface Testing
➢ Navigation Testing
➢ Component Testing
➢ Configuration Testing
➢ Performance Testing
➢ Security Testing.

Dr. Asmau Usman, Nile University of Nig.

 Testing Guidelines
➢ MobileApps that execute entirely on a
mobile device can be tested using
traditional software testing methods or using
emulators running on personal computers.
➢ On the other hand, thin-client MobileApps
that make use of server-based resources
are particularly challenging to test.
➢ In addition to many of the testing
challenges presented by WebApps, the
testing of thin-client MobileApps must also
consider transmission of data through
Internet gateways and telephone networks.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Users expect MobileApps to be context

aware and deliver personalized user
experiences based on the physical
location of a device in relation to available
network features.
➢ But testing Mobile-Apps in a dynamic ad
hoc network environment using every
possible device and network configuration
is difficult, if not impossible.
➢ To understand the objectives of MobileApp
testing, you should consider the many
unique challenges facing app designers.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ MobileApps are expected to deliver much of

the complex functionality and reliability
found in PC-based applications, but they are
resident on mobile platforms with relatively
limited resources.
➢ The following guidelines provide a basis for
mobile application testing:
➢ Understand the network and device
landscape before testing to identify
bottlenecks.
➢ Conduct tests in uncontrolled real-world test
conditions (field-based testing), especially for
a multitier mobile application.
13
Dr. Asmau Usman, Nile University of Nig.
Cont.,

➢ Select the right automation test tool.

➢ Ideally, the tool should support all desired
platforms, allow testing for various screen
types, resolutions, and input mechanisms—
such as touchpad and keypad, and
implement connectivity to the external
system to carry out end-to-end testing.
➢ Check the end-to-end functional flow in
all possible platforms at least once.
When Web services are involved, it is
difficult to trace the actual network path
required to deliver a MobileApp function
without performance tools.

Dr. Asmau Usman, Nile University of Nig.

Cont.,

➢ Conduct performance testing, GUI

testing, and compatibility testing using
actual devices.
➢ Even though these tests can be done
using emulators, testing with actual
devices is recommended.
➢ Measure performance only in realistic
conditions of wireless traffic and user
load.

14
Dr. Asmau Usman, Nile University of Nig.
THE TESTING STRATEGIES

Dr. Asmau Usman, Nile University of Nig.

The Testing Strategies
➢ Technology alone is not sufficient to guarantee
commercial success of a MobileApp.

➢ Users abandon MobileApps quickly if they do not

work well or fail to meet expectations.

➢ It is important to recall that testing has two

important goals:
➢ to create test cases that uncover defects
early in the development cycle and
➢ to verify the presence of important quality
attributes.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Developing a MobileApp testing strategy

requires an understanding of both software
testing and the challenges that make
mobile devices and their network
infrastructure unique.
➢ In addition to a thorough knowledge of
conventional software testing approaches.
➢ A MobileApp tester should have a good
understanding of telecommunications
principles and an awareness of the
differences and capabilities of mobile
operating systems platforms.

Dr. Asmau Usman, Nile University of Nig.

Cont.,

➢ This basic knowledge must be

complemented with a thorough
understanding of the different types of
mobile testing (e.g., MobileApp testing,
mobile handset testing, mobile website
testing), the use of simulators, test
automation tools, and remote data
access services (RDA).

Dr. Asmau Usman, Nile University of Nig.

 Are Conventional Approaches
Applicable?
➢ A comprehensive MobileApp testing
program includes the generic spiral
approach discussed in previous lectures, but
will also include adaptations discussed for
client-server architectures, real-time
computing, graphical user interfaces,
WebApps, and object-oriented systems.

➢ MobileApp testing also has unique

challenges that should be addressed to
ensure that an app meets both its functional
and non-functional requirements.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,
➢ It was suggests that MobileApp testers adapt the
strategy used for testing WebApps.
➢ Content must be tested to be sure that it was
chosen with the limitations of mobile devices
and ad hoc networks in mind.
➢ Compatibility testing and deployment testing
are more challenging in the mobile world, owing
to the large variety in device characteristics and
user environments.
➢ Performance testing needs to determine
whether the limited storage, processing,
connectivity, and power available on a mobile
device may negatively impact features or
functionality.

15
Dr. Asmau Usman, Nile University of Nig.
Cont.,

➢ MobileApp performance testing is often

conducted at a level of detail that is
seen only in the development of real-
time systems.
➢ MobileApps are often designed to be
used by people with less technical
knowledge than the typical Web user
precipitating the need for more
extensive testing of the user experience.

Dr. Asmau Usman, Nile University of Nig.

 The Need for Automation

➢ A MobileApp tester often encounters many

configuration variants (devices, operating systems,
and mobile networks) that may need to be
included to ensure that a MobileApp is both
functional and context aware.
➢ Because it is important to test a MobileApp
efficiently and completely, automated testing can
be useful for configuration testing or regression
testing.
➢ It should be noted, however, that it may not be
possible to automate all parts of MobileApp testing
(e.g., user interactions found in a handheld video
game).

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Automated testing tools can improve

team morale when testers would
otherwise be forced to process a large
number of repetitive test cases
mechanically.

➢ The availability of automated test tools

can encourage earlier and more
frequent testing, allowing the early
discovery of MobileApp defects.

Dr. Asmau Usman, Nile University of Nig.

 Testing in a Production
Environment

➢ Many MobileApp developers advocate

testing-in-the-wild, or testing in the users’
native environments with the production
release versions of the MobileApp
resources.
➢ Testing-in-the-wild is designed to be agile
and respond to changes as the
MobileApp evolves.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Some of the characteristics of in-the-wild

testing include adverse and
unpredictable environments, outdated
browsers and plug-ins, unique hardware,
and imperfect connectivity (both Wi-Fi
and mobile carrier).

➢ In order to mirror real-world conditions,

the demographic characteristics of
testers should match those of targeted
users, as well as those of their devices.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,
➢ Cloud-based testing can offer a standardized
infrastructure and preconfigured software
images, freeing the MobileApp team from
the need to worry about finding servers or
purchasing their own licenses for software
and testing tools.
➢ Cloud service providers give testers access to
scalable and ready to user virtual
laboratories with a library of operating
systems, test and execution management
tools, and storage necessary for creating a
test environment that closely mirrors the real
world.

Dr. Asmau Usman, Nile University of Nig.

CONSIDERING THE SPECTRUM OF
USER INTERACTION

Dr. Asmau Usman, Nile University of Nig.

 Considering The Spectrum of User
Interaction

➢ In a crowded marketplace in which

products provide the same functionality,
users will choose the MobileApp that is
easiest to use.
➢ The user interface and its interaction
mechanisms are visible to the MobileApp
users.
➢ It is important to test the quality of the user
experience provided by the MobileApp to
ensure that it meets the expectations of its
users.

Dr. Asmau Usman, Nile University of Nig.

 Cont.,

➢ Many of the procedures for assessing the

usability of software user interfaces can be
used to assess MobileApps.
➢ Similarly many of the strategies used to
assess the quality of WebApps may be
used to test the user interface portion of
the MobileApp.
➢ There is more to building a good
MobileApp user interface than simply
shrinking the size of a user interface from
an existing personal computer application.

Dr. Asmau Usman, Nile University of Nig.

 The characteristics of Mobile App

➢ The characteristics of Mobile App usability

become the focus of testing and these are:-
➢ Gesture Testing (e.g., swiping, zooming,
scrolling, selection).
➢ Voice Input and Recognition
➢ Virtual Key Board Input
➢ Alerts and Extraordinary Conditions (e.g., a
Wi-Fi signal may be lost, or an incoming text
message, phone call, or calendar alert).

Dr. Asmau Usman, Nile University of Nig.

 Testing Apps on Android
➢ Testing your app is an integral part of the app
development process.
➢ By running tests against your app consistently
➢ You can verify your app's correctness, functional
behavior, and usability before you release it
publicly.
➢ Testing also offers the following advantages:
➢ Rapid feedback on failures.
➢ Early failure detection in the development
cycle.
➢ Safer code refactoring, allowing you to
optimize code without worrying about
regressions.
➢ Stable development velocity, helping you
minimize technical debt.
Dr. Asmau Usman, Nile University of Nig.
 Testing Tools and Environments
➢ There are many ways to classify tests. However, the
most important distinction for app developers is
where tests run.
➢ Instrumented versus local tests.
➢ You can run tests on an Android device or on
another computer:
➢ Instrumented tests run on an Android device, either
physical or emulated.
➢ The app is built and installed alongside a test app
that injects commands and reads the state.
Instrumented tests are usually UI tests, launching an
app and then interacting with it.
➢ Local tests execute on your development machine
or a server, so they're also called host-side tests.
They're usually small and fast, isolating the subject
under test from the rest of the app.
Dr. Asmau Usman, Nile University of Nig.
Dr. Asmau Usman, Nile University of Nig.
 Summary

➢ MobileApp testing focuses on quality elements

such as content, function, structure, usability, use
of context, navigability, performance, power
management, compatibility, interoperability,
capacity, and security.
➢ The MobileApp testing strategy exercises each
quality dimension by initially examining “units” of
content, functionality, or navigation. Once
individual units have been validated, the focus
shifts to tests that exercise the MobileApp as a
whole.
➢ Finally, MobileApp testing should address
performance issues such as power usage,
processing speed, memory limitations, ability to
recover from failures, and connectivity issues.

27
Dr. Asmau Usman, Nile University of Nig.
 Practice Questions

 How can a MobileApp’s ability to take context

into account be tested?
 Are there any situations in which MobileApp
testing on actual devices can be disregarded?
 In your own words, discuss the objectives for
testing a MobileApp.
 Compatibility is an important quality dimension.
What must be tested to ensure that compatibility
exists for a MobileApp?
 Is it fair to say that the overall MobileApp-testing
strategy begins with user-visible elements and
moves toward technology elements?
 Are there exceptions to this strategy?

Dr. Asmau Usman, Nile University of Nig.

NEXT L

Next Topic:

THANKS
Dr. Asmau Usman, Nile University of Nig.