USABILITY TESTING
Mirela Gutica
Comp 4952
Technical Programming
Computer Science Technology
BCIT
Studies done with real users
The study should be planed carefully
A usability report should be produced
USABILITY TESTS/STUDIES
Formative usability
Usability studies done along with the project
design and implementation
Measure effectiveness, productivity, satisfaction
with the purpose of updating the design
Questions to answer:
What issues were identified by users?
What worked well? Not so well?
What improvements were suggested by users?
USABILITY TESTS/STUDIES
GOALS (1)
Summative usability
Final usability study before launching the product
Measure to what extent the final product meets the
requirements and users’ needs
Questions to answer:
Does the product meet the goals of the project?
What is the overall usability of the product?
How is the product’s usability in comparison to similar
current products?
What improvement can be done in the future?
Improvements can be done in new releases of the product
USABILITY TESTS/STUDIES
GOALS (2)
Traditional moderated usability tests
In a lab
One or more moderators
One-on-one between a participant and a moderator
Online usability tests
Could be moderated, but usually are unmoderated
Online surveys
Focus groups
EVALUATION METHODS
Completion
Time to complete a task
Effort to complete a task
Comparison
Similar product(s)
Alternative design
Evaluation: e.g., give a task (e.g., find a certain information) and observe how
the user finds that information)
Frequency of use
Productivity (e.g., number of clicks, easiness to identify the next step, etc.)
Navigation
Impact of changes
Measurement
Noticeability
Usability of a critical point
USABILITY METRICS
The methodology that you will employ for data collection and
analysis will inform your approach, process and methods used to
analyze your data
There are three methodologies:
Quantitative
Data is collected from a large number of participants (30+) via surveys
or focus groups
Qualitative
Data is collected from fewer participants (6) via interviews and/or
observations
Mixed-methods
Mixed methods are effective and can combine two or more sources of
data
DATA COLLECTION AND
ANALYSIS
Executive summary
Purpose of the study
Participants
Method
Evaluation methods
Usability metrics that were collected
Location and settings
Duration
Etc.
Findings and recommendations
Discussion
Appendices
USABILITY REPORT
Include the questionnaire, survey, data collected, etc.
The purpose of the study (an explanation of why the study is being done).
The procedure being used for the study. This section should also include a time
expectation for the participant and the protocol for requesting a break.
If there will be any type of recording, who will see the recordings, and what happens
to the recording material when the testing is completed (not all studies involve
recordings).
A statement of confidentiality and how the anonymity of the participant is
preserved.
Any risks to the participant (in most usability studies there is minimal risk).
The fact that participation is voluntary and that the participant can withdraw at any
time with no penalty.
Whom to contact with questions and for any further information after the study and
a statement that initial questions about the testing have been answered satisfactorily.
(Dumas and Loring, 2008)
INFORMED CONSENT
GUIDELINES
Tri-Council Policy Statement: Ethical Conduct for
Research Involving Humans – TCPS 2 (2018)
https://
ethique.gc.ca/eng/policy-politique_tcps2-eptc2_2018.html
Research ethics policies at BCIT
https://www.bcit.ca/applied-research/research-support/resear
ch-ethics
/
RESEARCH ETHICS
