Cognitive Models

Material from
Authors of Human Computer Interaction
Alan Dix, et al
Overview

Cognitive models represent users of interactive systems

• hierarchical - user’s task and goal structure

• linguistic – user-system grammar
• physical and device – human motor skills
• architectural – underlie all of above
Cognitive models

• They model aspects of user as they interact:

• understanding
• knowledge
• intentions
• processing

• Common categorization:
• Competence – represent kinds of behavior expected of user
• Performance – allow analysis of routine behavior in limited applications
Goal and taskhierarchies

Solve goals by solving subgoals

- Mental processing as “divide-and-conquer”

produce report
gather data
. find book names
. . do keywords search of names database
…further sub-goals
. . sift through names and abstracts by hand
…further sub-goals
. search sales database
..further sub-goals
layout tables and histograms
..further sub-goals
write description
..further sub-goals
Issues for goal hierarchies
• Granularity
• Where do we start?
• Where do we stop – how far to subdivide?
• Get down to a routine learned behavior, not
problem solving the
- unit task
• Conflict
• More than one way to achieve a goal
• Treatment of error
Techniques
• Goals, Operators, Methods and Selection (GOMS)
• Cognitive Complexity Theory (CCT)
• can represent error behavior
GOMS
• Goals - what the user wants to achieve

• Operators- basic actions user performs(granularity)

• Methods - decomposition of a goal into sub goals/operators

• may be more than one way or method to do that

• Selection - means of choosing between competing methods (GOMS

attempts to predict)
GOMS example
GOAL: ICONIZE-WINDOW
[select
GOAL: USE-CLOSE-METHOD
MOVE-MOUSE-TO-WINDOW-HEADER
POP-UP-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-L7-METHOD
PRESS-L7-KEY]

For a particular user Sam:

Rule 1: Select USE-CLOSE-METHOD unless
another rule applies.
Rule 2: If the application is GAME, select
L7-METHOD.
GOMS as a measure of performance
• selection rules can be tested for accuracy against user traces

• stacking depth of goal structure can estimate STM

requirements

• good for describing how experts perform routine tasks

• not for comparing across tasks
• not for predicting training time
Cognitive Complexity Theory - CCT
- basic premises of goal decomposition
- provides more predictive power

Two parallel descriptions:

• User - production rules of the form:
ifcondition thenaction

• Device - generalized transition networks

covered under dialogue models
Example: editing with vi

Production rules are in long-term memory

- 4 rules in the text on page 425

User sees a mistake - Model contents of working memory as

attribute-value mapping

(GOAL perform unit task

(TEXT task is insert space)
(TEXT task is at 5 23)
(CURSOR 8 7)
Example: editing with vi
Rules are pattern-matched to working memory,
e.g.,
LOOK-TEXT task is at %LINE %COLUMN
is true, with LINE = 5 COLUMN = 23.

Four rules model inserting a space –1st one only one that
can fire:
SELECT-INSERT-SPACE //bind to location
INSERT-SPACE-DONE //finished - unbind
INSERT-SPACE-1 //move cursor
INSERT-SPACE-2 //hit insert key and space
Example: editing with vi
When fired, binds the LINE and COL to 5 and 23 respectively and
adds to working memory

(GOAL insert space)

(NOTE executing insert space)
(LINE 5)
(COLUMN 23)

Now INSERT-SPACE-1 will fire

Notes on CCT
• Rules don’t fire in order written, may repeat
• Parallel model – rules can fire simultaneously
• Novice versus expert style rules
• Error behavior can be represented
• Measures
• Depth of goal structure
• Number of rules (more means interface more difficult to learn)
• Comparison with device description
Problems with goal hierarchies
• description can be enormous
• a post hoc technique – risk is that it is defined by the computer
dialog and not user
• expert versus novice

• Simple extensions possible

• goal closure (makes sure subgoal satisfied)
• eg. ATM example
Linguistic notations
• User’s interaction with a computer is often
viewed in terms of a language.

• Backus-Naur Form (BNF)

• Task-Action Grammar (TAG)
BNF
• Very common notation from computer science
• A purely syntactic view of the dialogue

Basic syntax:
nonterminal ::= expression
An expression contains terminals and nonterminals
combined in sequence (+) or as alternatives (|).

Terminals lowest level of user behavior

CLICK-MOUSE, MOVE-MOUSE
Nonterminals ordering of terminals; higher level of
abstraction
select-menu, position-mouse
draw line ::= select line + choose points + last point
select line ::= pos mouse + CLICK MOUSE
choose points ::= choose one | choose one + choose points
choose one ::= pos mouse + CLICK MOUSE
last point ::= pos mouse + DBL CLICK MOUSE
pos mouse ::= NULL | MOVE MOUSE + pos mouse
 Measurements with BNF

• Number of rules or number of + and | operators

• Complications
• same syntax for different semantics
• reflects user’s actions, not user's perception of system responses
• enforcement of consistency in rules

• Extensions
• include “information-seeking actions” in grammar
• parameterized grammar rules
Task-Action Grammar - TAG

• Making consistency in language more explicit than in BNF

• Encoding user's world knowledge
• (eg. up is opposite of down)
• Accomplished by
• Parameterized grammar rules
• Nonterminals are modified to include additional semantic features
Consistency in TAG

In BNF, three UNIX commands would be

described as
copy ::= cp + filename + filename
| cp + filenames + directory
move ::= mv + filename + filename
| mv + filenames + directory
link ::= ln + filename + filename
| ln + filenames + directory
Consistency in TAG

• In TAG, this consistency of argument order can be made explicit

using a parameter, or semantic feature for file operations.
file op[Op] ::= command[Op]+ filename + filename |
command[Op]+ filenames + directory

command[Op = copy] ::= cp

command[Op = move] ::= mv
command[Op = link] ::= ln
Notes

• Ignore system output

• (there are extensions to BNF and TAG)

• Hierarchical and grammar-based techniques initially developed

when systems were mostly command-line or keyboard and
cursor based.
Physical and device models

• Based on empirical knowledge of human motor system

• User's task: acquisition, then execution.
• These models only address execution
• Models are complementary with goal hierarchies
• Models
• The Keystroke Level Model (KLM)
• Buxton's 3-state model
Keystroke Level Model - KLM

Six execution phase operators

Physical motor K keystroking
P pointing
H homing
D drawing
Mental M mental preparation
System R response

Times are empirically determined.

Te
xe
cut
e = TK+ T P + T H + TD +TM +
T
R
Example
GOAL: ICONISE-WINDOW
[select
GOAL: USE-CLOSE-METHOD
MOVE-MOUSE-TO-WINDOW-HEADER
POP-UP-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-L7-METHOD
PRESS-L7-KEY]
Models so far

GOMS – cognitive processing involved in deriving subgoals to

carry out a task to achieve a goal
CCT – distinction between LTM (rules) and STM (working
memeory)
Linguistic (BNF and TAG) – focus on syntactic
KLM – motor and mental operators
Architectural models
All of cognitive models make assumptions
about the architecture of the human mind.
• Problem spaces – behavior viewed as sequence of agent/
environment states (can predict erroneous behavior)
• Interacting Cognitive Subsystems
• provides model of perception, cognition, and action
• 9 subsystems (5 physical, 4 mental)
• view of user as information processing machine
• concerned with determining how easy particular procedures of action
sequences become
Last notes
• Cognitive models attempt to represent users as they interact
with the system
• Three categories – what were they?
• Most cognitive models do not deal with user observation and
perception.
• Some techniques have been extended to handle system output,
but problems persist.
• Issues:
• Level of granularity
• Exploratory interaction versus planning