Assignment No:7

Aim: Evaluation of Interface

Problem statement:

Students are expected to evaluate minimum of two products / software interface against known HCI
evaluation.

Evaluate UI designed for CMS assignment using Nielsen’s Heuristic evaluation technique :

Nielsen's heuristics Rules:

1) Visibility of system status
2) Match between system and the real world
3) User control and freedom
4) Consistency and standards
5) Error prevention
6) Recognition rather than recall
7) Flexibility and efficiency of use
8) Aesthetic and minimalist design
9) Help users recognize, diagnose, and recover from errors
10) Help and documentation

Evaluate any UI / product using Ben Shneiderman’s eight golden rules for interface design.

Shneiderman's Eight Golden Rules of Interface Design

1. Strive for Consistency
2. Enable Frequent Users to Use Shortcuts
3. Offer Informative Feedback
4. Design Dialog to Yield Closure
5. Offer Simple Error Handling
6. Permit Easy Reversal of Actions
7. Support Internal Locus of Control
8. Reduce Short-Term Memory Load.
Theory:

Evaluation in HCI It is difficult to imagine that an interactive product could be designed and built
without any user-focussed evaluations taking place. It is not possible to pick up a HCI textbook that
does not include at least a chapter on the evaluation of user interfaces. The way in which these
usability evaluation methods are presented and classified varies between authors. In 1994, Nielsen
claimed that there were four basic ways of evaluating user interfaces; these being
• Automatically
• Empirically
• Formally
• Informally

He went on to suggest that automatic and formal evaluations were both problematic, and suggested
that only empirical and informal methods were really useful (Nielsen 1994). This has become the
accepted viewpoint in the usability community. Usability Evaluation Methods can generally be
described as either empirical or informal using Nielsen‟s words. The following table shows a list of
Usability Evaluation methods, categorized in this way.

Empirical Methods Informal Evaluations

User Walkthrough Heuristic Evaluation

Focus groups Expert reviews

Structured observations Cognitive walkthroughs

Cooperative evaluations Predictive modelling – GOMS

Activity logging Guidelines review

Data logging Consistency inspection

Observations Critical event analysis

Questionnaires Dialogue Error Analysis

Interviews Usability testing

 Controlled user tests

Physiological data analysis

Different authors divide evaluation in different ways. In User Interface Design, they are broken into
formative and summative methods (Le Peuple and Scane 2003). Formative evaluations take place
early in the design process and the results of these evaluations inform the later design.
(Shneiderman 1998)divides evaluation by considering who the evaluator is. This results in expert
reviews and novice reviews. (Faulkner 1998) compares analytical and statistical methods. In John
Carrolls design,, the issues of evaluation of learner centred interfaces is touched on.

How Evaluation methods are evaluated :

Given the large number of methods available, and the many different ways of classifying them, it
can be difficult to know which methods should be used. There is an abundance of literature on the
evaluation of evaluation methods.

Work by (Nielsen and Phillips 1993) focussed on GOMS, Heuristics and User testing. These were
used with three different views of the design, cold, warm and hot. The cold testing took place with
just a written specification, the warm test was carried out with a limited time on a prototype (about
an hour), and for the hot test, testers could play on the prototype for as long as they wanted. The
user testing activity was only used with the end product. It was a within subjects design and subjects
were allowed to practice until they plateau-ed. They had a sequence of tasks to do during which
error messages and feedback were given and this activity was followed by a subjective satisfaction
questionnaire.

The findings were that the estimates between experts doing GOMS and Heuristic evaluations were
very varied. This suggested that it is best to not rely on a single inspection. Other authors have made
similar observations. (REFS) The cost of a heuristic evaluation with a hot view of the design was
costlier than the user test and it was remarked that unless the product is unstable, or users are
unavailable, the user test was preferred at this stage. (Savage 1996) compared expert reviews, user
reviews and user testing. In this study, expert reviews were defined to be inspection methods carried
out by human factors specialists. These included heuristic evaluations, cognitive and pluralistic
walkthroughs, and consistency and standards inspections. The usability tests were conducted in a
role-play interaction using a talk aloud session. User reviews involved potential end users in
viewing slide shows of the product and completing questionnaires and engaging in group
discussion. Results from this study were that expert reviews tended to inform user interface issues
that needed more research with end users; the other two methods flagged up design issues. Heuristic
evaluations and user tests have been shown in some studies to identify discrete sets of usability
problems (Law and Hvannberg 2002).

In industry, user testing, where users are brought into a lab and asked to think aloud while
performing tasks, and are later questioned about their experience of the software, is the most widely
used technique (Nielsen and Mack 1994). Cognitive walkthroughs (Wharton, Rieman et al. 1994),
Heuristic Evaluations (Nielsen 1994) and GOMS (Card, Moran et al. 1983) are all more economical
as they do not require running a prototype or actual users. Empirical methods rely on the availability
of real users to test the interface, whereas informal evaluations rely on the skill and experience of
the evaluator. The recommended number of evaluators for a heuristic evaluation is 3 – 5 and
Nielsen has claimed that five subjects are enough for a usability test as well. In some instances
where users are scarce, users may need to be saved for a user test, thus forcing the need for expert
reviews. (Nielsen 1994) .

However, (John 1996) points out that there are many questions that remain about all these
techniques including :

• Is the technique real world enough?

• Does it find faults and fixes, or just faults?
• When in the development process should it be used?
• How does it fare in a cost benefit analysis?
• How can techniques be best used together?
• What are the costs of learning a new technique?

Designing an Evaluation Strategy :

Determinants (Shneiderman 1998)

• Stage of design
• Novelty of project
• Number of expected users
• Criticality of the interface
• Costs of product and finances available for testing
• Time available
• Experience of the design and evaluation team

It is possible to simplify these into four stages; these being.

• Purpose of the product – users would be defined here
• Availability of resources
• Stage of the project
• Purpose of the evaluation

These dimensions are described in the following section :

Purpose of Evaluation The Evaluation will have its‟ own purpose, it may be to
• Predict problems
• Discover problems
• Evaluate against another product This may change over a product design lifecycle. It is possible
that the evaluator may want one evaluation to cover all three aspects.
Google Slides / Docs etc. :
Notion :

Evaluating Google software and Notion (Note Taking software) :

Google Slides / Docs etc. Notion

1. Strive for Consistency

Consistent layout and design with other Basic block structure is consistent .Core UI
Google Workspace apps. Fonts, tools, and elements remain recognizable.
colors are uniform across all slides and
devices.

2. Enable Frequent Users to Use Shortcuts

Offers various keyboard shortcuts (e.g., ctrl+z, Provides multiple shortcuts (e.g., / for
ctrl+c). Power users can navigate quickly commands). Users can customize shortcuts
between slides, format, and present using and use commands to quickly format,
shortcuts. navigate, and manage tasks.
3. Offer Informative Feedback

Provides immediate feedback on actions like Clear feedback on actions such as edits and
inserting images, real-time collaboration, and database changes. Collaboration is visible in
changes . real-time, and actions are synced with minimal
delay.

4. Design Dialog to Yield Closure

Clear sequence when completing tasks like Clear feedback when tasks like creating
saving, sharing, and presenting. Users get databases or sharing pages are completed.
feedback when the task is done, such as Users are notified of action completion via
publishing a slideshow. alerts or changes.

5. Offer Error Prevention and Simple Handling

Error messages are simple and informative, Handles errors like failed syncs with clear
like for connectivity or saving issues. Provides messages. The app ensures minimal disruption
suggestions to resolve errors (e.g., reconnect by saving changes locally and syncing later.
message).

6. Permit Easy Reversal of Actions

Undo/redo options are available. Version Undo/redo options are available, and users can
history allows reverting to older versions of revert to previous versions of pages or blocks.
the presentation Version history is also available for data
recovery

.
.

7. Support Internal Locus of Control

Users control formatting, collaboration, and Notion gives users control over how content is
presentation flow. They can manage settings organized (blocks, databases). Permissions and
like access permissions and slide navigation. access controls give users authority over their
workspace.

8. Reduce Short-Term Memory Load

Clear, intuitive menus and toolbars reduce Notion’s block system helps break down
memory load. The autosave feature and complex tasks. However, its complexity can
collaboration tools prevent users from having lead to slightly higher memory load compared
to remember tasks. to simpler tools.
Conclusion :

Thus we have evaluated two software interface against known HCI evaluation.