Download to read offline
Uploaded byAnne Petersen
Usability testing in the open
This document summarizes Anne Petersen's presentation on usability testing in the open. The presentation: 1) Had attendees participate in a usability testing exercise where they tested instructions in different roles of interviewer, observer, and interviewee. This was done to demonstrate the method. 2) Discussed how usability testing is a human-centered design method that involves testing designs with real users to identify problems and improve the design. 3) Explained that government agencies like 18F conduct user research and usability testing in the open on platforms like GitHub to gather feedback and improve designs and services for the public.
More Related Content
Similar to Usability testing in the open
Usability testing in the open
- 1. Usability Testing inthe Open Anne Petersen Director of Experience Design, 18F
- 2.
- 3. Hi. Anne Petersen anne.petersen@gsa.gov Director ofExperience Design 18F they/them @apetersen
- 4. I’m from thegovernment, and I’m here to help. The nine most terrifying words in the English language, according to Ronald Reagan
- 5.
- 6. 1 2 3 Agenda Exercise: Tryusability testing What did we just do? Introduction to user-centered design Exercise: What can you use in your work?
- 7.
- 8. Form teams of3 and assign roles: Interviewer, Observer, and Interviewee. If you have too many people, 2 notetakers. EXERCISE
- 9. Interviewer: You are workingto improve the instructions in the handout. Ask questions, but... NO HELPING! EXERCISE
- 10. Observer: Write down anyreactions, points of confusion, and pain points you observe. Again: NO HELPING! EXERCISE
- 11. Interviewee: Do your bestand think out loud. EXERCISE
- 12. This is arace. We’ll celebrate the fastest team. You have 10 minutes, starting now. Go! EXERCISE
- 14.
- 15.
- 16. Come up with3 hypotheses about how you could improve the instructions. EXERCISE
- 17. Now, let’s switchroles and go again. Instead of fastest, we’ll celebrate the loveliest. EXERCISE
- 19.
- 20.
- 21. What went right? Whatwent wrong? How would you improve the experience? EXERCISE
- 22.
- 23. What did wejust do?
- 24. Usability testing ...which isa human-centered design method
- 25.
- 26. Design is decidinghow a thing should be.” I - William Van Hecke “
- 27.
- 28. Design is decidinghow interactions should work. I
- 29. Oxo Good Grip Peeler Designproblem: standard kitchen utensils can be uncomfortable and hard to use, especially for people with arthritis or other grip issues. Handle is soft and comfortable, with flex fins to adjust for additional pressure. Non-slip, even when hands are wet or oily. During the design phase, designers tested hundred of handles to find the one that was the most comfortable and functional. The blade holds up for years and slices through even tough fruit and vegetable skin. Built-in potato eyer. *Generic peeler
- 30. Think of somethingthat is intentionally designed. ACTIVITY
- 31. 1/ Good designis user-centered
- 32. User-centered design helpsus design for actual humans.* * instead of hypothetical ones
- 33. User-centered design helpsus design for actual humans who will use what we’re making.
- 34. Drivers need toknow where and when they are allowed to park on the street. SCENARIO
- 35. Badly designed* solution tothe problem. *not user-centered
- 36. Better designed* solution tothe problem. *user-centered
- 37. How do designersknow what decisions to make?
- 38.
- 39. People* are badat predicting how users* will behave. * even designers * even when they are the user
- 40. 1/ Research helpsus understand users’ lives and needs. How do they work now? What do they need? How do they think about their work? How do they feel about it? What gets in their way?
- 41. 2/ Research helpsus understand how users experience what we design. Does what we’ve built make it easier? Where do they get stuck? What’s confusing? What makes them frustrated? What saves time?
- 42. We want toimprove how people get permits to cut down their own Christmas trees. SCENARIO
- 43. Maybe we shouldbuild a website! Then they can access permits online, and Forest Service employees can verify them using their phones.
- 44. 1. Observe: goto the national forests and watch how people get permits and how staff check permits.
- 45. Maybe we shouldbuild a website! There’s no reception in the forest where the permits will be used.
- 47. All the ideas!Fewer, better ideas The best, tested, proven idea
- 48. 2. Build something:come up with an idea and build a version to test.
- 49. Printable permits shouldwork better. People can pay online and get the permit immediately, instead of making time to go to an office.
- 51. 3. Test andadjust: watch people use what you’ve made, and make it better based on what happens.
- 53. This year location wasnot very readable Needed to find a way for permits to cover more than 1 tree over more than 1 day The permit number itself wasn’t very important to users or enforcement officers Turned out the forest name wasn’t super important to enforcement officers, since people were unlikely to have a permit for the wrong forest
- 55. Even if usersdon’t have internet access, all the instructions are right here—including the size of tree they’re allowed to have! Important for enforcement officers to know how many trees are allowed at-a-glance — they’re not supposed to stop cars, so they have to see it while they’re passing by Needed to find a way for permits to cover more than 1 tree over more than 1 day
- 56. ● Friendly, opensource, interactive site facilitating responsible forest access ● Pilot: Pay and print-at-home Christmas tree permit ● Launched November 2018 in four pilot forests openforest.fs.usda.gov Christmas tree permitting module Introducing Open Forest
- 57.
- 58. github.com/USDAForestService/fs-open-forest/wiki Public wiki ● Projectroadmap ● Goals and risks ● Onboarding materials ● Research findings ● Technical Information ● Acquisitions ● “How we work” Open documentation
- 59. Working in theopen
- 60. Blogs on 18F 18f.gsa.gov/blog- 18f.gsa.gov/what-we-deliver/forest-service ● Help users learn ● Help the team learn Blog posts
- 61. Tl;dr: We mayhave saved Christmas. Heat Miser The Year Without A Santa Claus
- 62.
- 63. The best toolfor resolving disputes within a design team, for making design decisions based on data rather than opinion, is sitting next to someone who is a real person who wants to accomplish something as they use your design to do it. — Dana Chisnell (Usability expert, formerly U.S. Digital Service)
- 64.
- 65. User-centered design helpsus design for changing humans.
- 66. START WITH PROBLEM LEARN USER NEEDS ALIGN WITH BUSINESS NEEDS/ CONSTRAINTS PROTOTYPE / BUILD TESTWITH/ GIVE TO USERS REPEAT How? User-centered, iterative processes
- 67. Design is deciding, overand over, how interactions should work for humans based on research.
- 68. Design is deciding, overand over, how interactions should work for humans based on research.
- 69. Design is deciding, overand over, how interactions should work for humans based on research.
- 70. Design is deciding, overand over, how interactions should work for humans based on research.
- 71. The 18F Guides Thetechnical guides that 18F uses in our work with other agencies. Built on user-centered development, testing to validate hypotheses, shipping often, and deploying products in the open, they help us do our day-to-day work and are public domain for anyone to use. 18f.gsa.gov/guides
- 73. Research question(s) Whoyou hope to talk to How you plan to recruit people The methods you’ll use Elements of a good research plan 1 2 43
- 74. Characteristics of agood research question The question is feasible. The question is clear. The question is significant. The question is ethical.
- 75. “The result offormulating questions is it helps you to circumscribe a specific area of a more or less complex field which you regard as essential.” Agee, Jane. "Developing qualitative research questions: a reflective process." International Journal of Qualitative Studies in Education 22.4 (2009): 431-447.
- 76. 1 2 34 5 Recruiting priorities Target users with an urgent need Target users with recent experience People who fit your audience Friends and those in your network Team members Most reliable results Least reliable
- 77. What can youuse?
- 78. What do youdesign that other people use?
- 79. How would youlearn more about how someone uses something you make? Exercise:
- 80. ● What doyou make? ● Who uses it? ● What questions do you have about how people use what you make?
- 81.
- 82.
- 83.
- 84. A few thingsdesigners do 84 ● Discover what users want a system to do ● Discover what users need a system to do ● Help the team understand users’ underlying goals and human motivations ● Create ideas for new products or services ● Sketch ideas for new products and services to help others understand them ● Design systems that are easy for users to understand and learn ● Create interactions that lead users to a desired action or thought process ● Prioritize users’ needs ● Run usability tests to understand how well users can understand and use a system ● Use expert analysis to identify usability problems before testing ● Make it easy for users to notice, and fix, errors and bad data ● Create prototypes to test ideas before investing lots of development effort ● Shape how systems look and act to convey a specific personality, tone, or atmosphere ● Shape how language is used to convey a specific personality, tone, or atmosphere What do designers do?
- 85. A few thingsdesigners don’t do 85 ● Ensure the product vision is fulfilled ● Confirm your preconceived notions, even when they’re inaccurate ● Make the design “pop more” ● Make the design “pretty” ● Draw the ideas in your head ● Read minds ● Research every single decision, big or small ● Add UI elements because they were asked to ● Ignore end user needs ● Follow numbers blindly ● Work alone ● “Throw things over the wall” to developers ● Fix the database ● Architect software ● Make the words sparkle ● Fetch coffee What do designers do?