KEMBAR78
User testing workshop | PDF
USER TESTING 
WORKSHOP 
Joe Dollar-Smirnov @fruitskin 
Director of UX, Red Badger & Lead Instructor, General Assembly
USER TESTING WORKSHOP 
OBJECTIVES 
‣ Set up scenario 
‣ Run a usability test 
‣ Record feedback
USER TESTING WORKSHOP 
APPROACH
USER TESTING WORKSHOP 
WHAT DOES IT TAKE? 
‣ Bravery 
‣ Humble Pie
USER TESTING WORKSHOP 
TOOLS 
‣ Analytics Tools - GA, Web Trends, Optimal Workshop, Qubit, Crazy Egg, 
Inspectlet the list goes on… 
‣ Silverback, Camtasia, Screenflow 
‣ Webcam, GoPro 
‣ Paper, Pens and Empathy
USABILITY & USER TESTING 
METHODS 
‣ Formal 
‣ Guerilla (Cafe Test) 
‣ Remote 
‣ Unmoderated 
‣ Face to Face
USABILITY & USER TESTING 
METHODS 
‣ Formal 
‣ Guerilla (Cafe Test) 
‣ Remote 
‣ Unmoderated 
‣ Face to Face
USABILITY & USER TESTING 
METHODS 
‣ Formal 
‣ Guerilla (Cafe Test) 
‣ Remote 
‣ Unmoderated 
‣ Face to Face
USABILITY & USER TESTING 
METHODS 
‣ Formal 
‣ Guerilla (Cafe Test) 
‣ Remote 
‣ Unmoderated 
‣ Face to Face
USABILITY & USER TESTING 
METHODS 
‣ Face-to-face 
‣ This is the recommended method 
‣ Provide users with tasks to complete on a prototype 
‣ Encourage them to talk out loud as they move through 
interactions 
‣ Observe their actions
USABILITY & USER TESTING 
PREPARATION 
‣ Create task scenarios 
‣ Should describe the task in a clear and unambiguous way 
‣ Written in the user’s language (don’t use jargon) 
‣ Don’t prompt the solution
USABILITY & USER TESTING 
PREPARATION 
‣ Create a discussion guide 
‣ Should keep the participant talking (think aloud protocol) 
‣ You are trying to learn without leading 
‣ Keep the feedback flowing 
‣ Prepare to note down reactions not just words 
‣ “What would you expect” not “Do you like?”
USABILITY & USER TESTING 
OUTPUTS 
‣ Notes 
‣ Video, Audio 
‣ Usability Labs will have highly finished reports 
‣ Unmoderated tests - data reports and recorded sessions 
‣ Report good and bad, don’t make conclusions (yet), stick to 
what you see 
‣ SUS Score?
USABILITY & USER TESTING 
THE TESTS 
‣ QUIS - Questionnaire for User Interaction Satisfaction 
‣ SUMI - Software Usability Measurement Inventory 
‣ PSSUQ - Post-Study System Usability Questionnaire 
‣ SUS - Software Usability Scale 
‣ ASQ - After-Scenario Questionnaire 
‣ ER - Expectation Ratings 
‣ UME - Usability Magnitude Estimation 
‣ SEQ - Single Ease Question 
‣ SMEQ - Subjective Mental Effort
USABILITY & USER TESTING 
SUS 
‣ Since mid-80s by John Brooke 
‣ Benchmark and Summative 
‣ Self Described as “Quick and Dirty” 
‣ Scale of 1 - 5 (strongly disagree - strongly agree)
USABILITY & USER TESTING 
THE QUESTIONS 
1. I think that I would like to use this system frequently. 
2. I found the system unnecessarily complex. 
3. I thought the system was easy to use. 
4. I think that I would need the support of a technical person to be able to use this system. 
5. I found the various functions in this system were well integrated. 
6. I thought there was too much inconsistency in this system. 
7. I would imagine that most people would learn to use this system very quickly. 
8. I found the system very cumbersome to use. 
9. I felt very confident using the system. 
10. I needed to learn a lot of things before I could get going with this system.
EXERCISE 
KEY OBJECTIVE(S) 
TIMING 
1 min 
1 min 
5 min 
5 min 
DELIVERABLE 
ACTIVITY 
Write scenarios for your test that allows you to carry out usability testing. 
Scenarios 
1. Split group and choose website/app or per group 
2. Get in to pairs 
3. Create a scenario 
4.Discuss
EXERCISE 
KEY OBJECTIVE(S) 
TIMING 
2 min 
5 min 
DELIVERABLE 
ACTIVITY 
In your pairs write a testing discussion guide. 
1.Write a testing discussion guide as a group 
2. Refine for your own needs 
Usability testing discussion guide
EXERCISE 
KEY OBJECTIVE(S) 
TIMING 
1 min 
8 min 
1 min 
10 min 
DELIVERABLE 
ACTIVITY 
Carry out a usability test , take notes and ask the participant to fill in 
a Software Usability Scale questionnaire 
None. 
1.Choose another pair 
2. Tests project on the each other (take notes) 
3.Ask participant to complete SUS 
4. Swap and repeat
EXERCISE 
KEY OBJECTIVE(S) 
TIMING 
5 min 1. Do it on the board 
DELIVERABLE 
ACTIVITY 
Calculate SUS as a group 
None.
USABILITY & USER TESTING 
THE MATHS 
‣ Odd numbered questions (x-1) 
‣ Even numbered questions (5-x) 
‣ Sum * 2.5 = Score out of a 100 (not a percentage) 
‣ Average is 68 across over 5000 test results in different studies
USER TESTING WORKSHOP 
FURTHER READING 
‣ Quantifying the User Experience, J. Sauro & J. R. Lewis
USER TESTING WORKSHOP 
THANKS! 
Q&A

User testing workshop

  • 1.
    USER TESTING WORKSHOP Joe Dollar-Smirnov @fruitskin Director of UX, Red Badger & Lead Instructor, General Assembly
  • 2.
    USER TESTING WORKSHOP OBJECTIVES ‣ Set up scenario ‣ Run a usability test ‣ Record feedback
  • 3.
  • 4.
    USER TESTING WORKSHOP WHAT DOES IT TAKE? ‣ Bravery ‣ Humble Pie
  • 5.
    USER TESTING WORKSHOP TOOLS ‣ Analytics Tools - GA, Web Trends, Optimal Workshop, Qubit, Crazy Egg, Inspectlet the list goes on… ‣ Silverback, Camtasia, Screenflow ‣ Webcam, GoPro ‣ Paper, Pens and Empathy
  • 6.
    USABILITY & USERTESTING METHODS ‣ Formal ‣ Guerilla (Cafe Test) ‣ Remote ‣ Unmoderated ‣ Face to Face
  • 7.
    USABILITY & USERTESTING METHODS ‣ Formal ‣ Guerilla (Cafe Test) ‣ Remote ‣ Unmoderated ‣ Face to Face
  • 8.
    USABILITY & USERTESTING METHODS ‣ Formal ‣ Guerilla (Cafe Test) ‣ Remote ‣ Unmoderated ‣ Face to Face
  • 9.
    USABILITY & USERTESTING METHODS ‣ Formal ‣ Guerilla (Cafe Test) ‣ Remote ‣ Unmoderated ‣ Face to Face
  • 10.
    USABILITY & USERTESTING METHODS ‣ Face-to-face ‣ This is the recommended method ‣ Provide users with tasks to complete on a prototype ‣ Encourage them to talk out loud as they move through interactions ‣ Observe their actions
  • 11.
    USABILITY & USERTESTING PREPARATION ‣ Create task scenarios ‣ Should describe the task in a clear and unambiguous way ‣ Written in the user’s language (don’t use jargon) ‣ Don’t prompt the solution
  • 12.
    USABILITY & USERTESTING PREPARATION ‣ Create a discussion guide ‣ Should keep the participant talking (think aloud protocol) ‣ You are trying to learn without leading ‣ Keep the feedback flowing ‣ Prepare to note down reactions not just words ‣ “What would you expect” not “Do you like?”
  • 13.
    USABILITY & USERTESTING OUTPUTS ‣ Notes ‣ Video, Audio ‣ Usability Labs will have highly finished reports ‣ Unmoderated tests - data reports and recorded sessions ‣ Report good and bad, don’t make conclusions (yet), stick to what you see ‣ SUS Score?
  • 14.
    USABILITY & USERTESTING THE TESTS ‣ QUIS - Questionnaire for User Interaction Satisfaction ‣ SUMI - Software Usability Measurement Inventory ‣ PSSUQ - Post-Study System Usability Questionnaire ‣ SUS - Software Usability Scale ‣ ASQ - After-Scenario Questionnaire ‣ ER - Expectation Ratings ‣ UME - Usability Magnitude Estimation ‣ SEQ - Single Ease Question ‣ SMEQ - Subjective Mental Effort
  • 15.
    USABILITY & USERTESTING SUS ‣ Since mid-80s by John Brooke ‣ Benchmark and Summative ‣ Self Described as “Quick and Dirty” ‣ Scale of 1 - 5 (strongly disagree - strongly agree)
  • 16.
    USABILITY & USERTESTING THE QUESTIONS 1. I think that I would like to use this system frequently. 2. I found the system unnecessarily complex. 3. I thought the system was easy to use. 4. I think that I would need the support of a technical person to be able to use this system. 5. I found the various functions in this system were well integrated. 6. I thought there was too much inconsistency in this system. 7. I would imagine that most people would learn to use this system very quickly. 8. I found the system very cumbersome to use. 9. I felt very confident using the system. 10. I needed to learn a lot of things before I could get going with this system.
  • 17.
    EXERCISE KEY OBJECTIVE(S) TIMING 1 min 1 min 5 min 5 min DELIVERABLE ACTIVITY Write scenarios for your test that allows you to carry out usability testing. Scenarios 1. Split group and choose website/app or per group 2. Get in to pairs 3. Create a scenario 4.Discuss
  • 18.
    EXERCISE KEY OBJECTIVE(S) TIMING 2 min 5 min DELIVERABLE ACTIVITY In your pairs write a testing discussion guide. 1.Write a testing discussion guide as a group 2. Refine for your own needs Usability testing discussion guide
  • 19.
    EXERCISE KEY OBJECTIVE(S) TIMING 1 min 8 min 1 min 10 min DELIVERABLE ACTIVITY Carry out a usability test , take notes and ask the participant to fill in a Software Usability Scale questionnaire None. 1.Choose another pair 2. Tests project on the each other (take notes) 3.Ask participant to complete SUS 4. Swap and repeat
  • 20.
    EXERCISE KEY OBJECTIVE(S) TIMING 5 min 1. Do it on the board DELIVERABLE ACTIVITY Calculate SUS as a group None.
  • 21.
    USABILITY & USERTESTING THE MATHS ‣ Odd numbered questions (x-1) ‣ Even numbered questions (5-x) ‣ Sum * 2.5 = Score out of a 100 (not a percentage) ‣ Average is 68 across over 5000 test results in different studies
  • 22.
    USER TESTING WORKSHOP FURTHER READING ‣ Quantifying the User Experience, J. Sauro & J. R. Lewis
  • 23.