MODULE 2: THE COGNITIVE PSYCHOLOGY AND PERCEPTION OF HCI

Introduction:

In the dynamic realm of Human-Computer Interaction (HCI), the intersection of cognitive psychology
and perception emerges as a fascinating and crucial domain. As we celebrate the first year of delving into
the intricacies of this multifaceted field, it is imperative to reflect on the profound impact it has had on shaping
our understanding of how humans interact with technology.
Cognitive psychology, with its emphasis on mental processes such as perception, memory, and
problem-solving, provides a robust framework for deciphering the complexities of human cognition in the
context of HCI. At the heart of this interdisciplinary synergy lies the exploration of how individuals perceive,
interpret, and respond to the ever-evolving landscape of digital interfaces.
Perception, a cornerstone of cognitive psychology, plays a pivotal role in HCI, influencing user
experiences, interface design, and overall usability. From the subtle nuances of color schemes to the
arrangement of interactive elements, every facet of HCI is intricately connected to the way users perceive
and process information. Understanding these perceptual mechanisms empowers designers and developers
to create interfaces that seamlessly align with users' cognitive processes, enhancing engagement and
usability.
Over the past year, our exploration of the cognitive psychology and perception of HCI has unfolded
through the lens of user-centered design, cognitive load theory, and the psychology of aesthetics. We have
delved into the intricacies of attention, memory, and decision-making, unraveling the mysteries of how these
cognitive processes shape users' interactions with digital interfaces.

Objective:

By the end of this lesson onThe Cognitive Psychology and Perception of HCI, students will be able to:

• Understanding how users encode, store, and retrieve information is crucial in designing interfaces
that facilitate efficient memory processes.
• Understanding attentional limitations helps in creating interfaces that guide users effectively,
prioritize essential information, and minimize distractions.
• Be able to identify the key characteristics of applications that are perceived as user-friendly design
interface.

Content:

Module 2: The Cognitive Psychology and Perception of HCI

⚫ Memory, attention, and mental models.

⚫ Perception and Sensation
⚫ Gestalt Principles
 Module 2: The Cognitive Psychology and Perception of HCI

Memory, Attention, and Mental Models.

Module1:
Introduction

In the intricate tapestry of Human-Computer Interaction (HCI), three foundational elements stand as
pillars, shaping the very essence of our digital experiences: Memory, Attention, and Mental Models. As we
mark a significant milestone in our exploration of HCI, it is fitting to delve into the profound interplay between
these cognitive cornerstones and their profound impact on the design, usability, and user experience of digital
interfaces.
Memory, a cognitive faculty often likened to a dynamic filing system, plays a pivotal role in HCI. From
the retention of past interactions to the recall of learned patterns, understanding the nuances of human
memory is key to creating interfaces that seamlessly align with users' cognitive processes. Our journey into
the realm of memory in HCI unveils not only the challenges posed by information overload but also the
opportunities for designing interfaces that facilitate efficient information retrieval and enhance overall user
satisfaction.
Attention, the gateway to conscious awareness, shapes the way users engage with digital interfaces.
The competition for attention in an increasingly digital landscape underscores the importance of designing
interfaces that capture and maintain users' focus. Our exploration of attention in HCI scrutinizes the factors
influencing selective attention, the role of visual and auditory stimuli, and the implications for crafting
interfaces that guide users through complex digital environments without overwhelming cognitive resources.
Mental Models, the cognitive frameworks through which individuals interpret and navigate the world,
are the linchpin of effective HCI. Understanding users' mental models empowers designers to create
interfaces that align with users' expectations, reducing cognitive friction and enhancing overall usability. As
we reflect on the past year's exploration, our investigation into mental models in HCI has unraveled the
intricate dance between user expectations and interface design, shedding light on strategies to bridge the
gap between the digital and cognitive realms.

What is Cognitive Psychology?

Cognitive psychology is the study of mental
processes such as attention, perception, memory, and
problemsolving. Understanding thesenprocesses is crucial for
designing effective user interfaces.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

What is memory, attention, and mental models?

Memory is the cognitive process by which information is encoded, stored, and retrieved for later
use. It involves the retention of past experiences, learning, and knowledge.
Types of Memory:
1. Short-term Memory (STM): Holds a limited amount of
information for a short duration.
2. Long-term Memory (LTM): Stores information for an
extended period, potentially for a lifetime.
3. Working Memory: The system that actively holds and
manipulates information needed for cognitive tasks.

Attention is the cognitive process of selectively concentrating

on one aspect of the environment while ignoring other stimuli. It
involves the allocation of mental resources to specific information.

Types of Attention:
1. Selective Attention: Focusing on a specific stimulus while filtering out others.
2. Divided Attention: Allocating attention to multiple stimuli simultaneously.
3. Sustained Attention: Maintaining focus over an extended period.

Mental models are cognitive representations or frameworks that individuals construct to understand how
something works or to predict the outcome of their actions. They influence perception, decision-making, and
problem-solving, in short it is an explanation of someones thought process about how
1. Dynamic: Mental models evolve as individuals gain new experiences or information.
2. Schematic: They simplify complex systems by representing key elements and their relationships.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

The mental model has two key components

Mental models are cognitive representations or frameworks that individuals construct to understand
how something works or to predict the outcome of their actions. These mental models help people make
sense of the world around them and guide their interactions. By simply understanding these components of
mental models is crucial for designers in the field of Human-Computer Interaction (HCI). By aligning interfaces
with users' mental models, designers can enhance usability and user experience, making interactions more
intuitive and reducing cognitive load.
⚫ The information architecture: How do the various elements used within an application relate to each
other?
⚫ The process flow: Which steps does the user of an application have to perform to accomplish a task
and what states do the elements of the application move in between? What business rules apply?

Lets dive into the components of mental models and see how a mental model can give a clear prespective
about the process. Lets have social media as an example of a process that we would like to create a mental
models for specific the newsfeedd of social media.
News Feeds
In social media newfeed no matter if it’s Facebook, LinkedIn, Twitter or Slack, newsfeeds are fundamentally
similar and pretty straightforward: items of interest are listed in chronological order in a continuous flow. Often
the user can interact with these items, such as by clicking on them to get more detail, “liking” them, re-
posting/sharing and replying.

Mental Model of Email Client

An email client organizes emails into an inbox in list format and commonly has different sub-
folders for organizing messages. The anatomy of an email message (information architecture)
is as follows:

A message has a subject, a sender, one or multiple recipients, and can also have CC:’s, BCC:’s as well as
attachments. Everyone who has ever used an email client is familiar with this mental model, which makes it
very straightforward to switch between email programs with only a limited learning curve.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

Do notice the language used in the context of email: mail, inbox, and folders. The icon typically associated
with attachment is the paper clip. All these are rooted in the physical world where actual paper mail gets
delivered. (This applies even to “CC:” which stands for carbon copy. Not much carbon happening in
cyberspace!) By using the same terms and concepts, a person familiar with postal mail can easily apply those
when learning about email.

Mental Model of Online Ordering/E-Commerce

Let’s look at another, a little more complex, common example: placing orders online via a consumer e-
commerce site. Anybody who’s ever placed an order with an online retailer is familiar with the following
structure:

From a process, perspective, here’s what we’re used to seeing:

All e-commerce sites use the same fundamental architecture and process flow. For good reason: this pattern
works. Users are familiar with it. There is no learning curve. Could online ordering be done differently? Maybe.
But if someone came up with a better, more streamlined way that breaks this mental model in a significant
way, the potential increase in efficiency may be far outweighed by the friction caused by users having to learn
a new mental model and getting used to a site that works very differently than all the other ones they’re used
to. This may not be worth it!
And once again, notice the terminology: shopping cart and check out. This sounds like a brick-and-mortar
store, right? With all the freedoms the online world offers us, application designers have still chosen to
perpetuate mental models from the physical world.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

Why Mental Models Matter

Users need a mental model that aids them in using an application. If their mental model matches what the
application is actually doing and how it’s designed and organized, users perceive it as easy to use and
intuitive. It takes little effort to learn it and become productive
When we create new applications or features, we should think hard about whether to invent brand new,
creative mental models or if existing ones can be effectively used or built upon. While it may be less “creative”
to go with a known pattern, it will greatly reduce the friction users will experience when confronted with your
new app or feature set which they’re not familiar with.
If no good mental model exists to build upon, we are required to design our application in such a way that
users can easily learn and form a new mental model of our application in their mind. The application should
be clearly organized, consistent, and make it at easily deduce the information architecture and process flows
required to get the user’s job done. That seems very redundant and obvious, however there are too many
examples of applications that seem convoluted, inconsistent and apply business rules that aren’t transparent
to the user.

Here are some exaggerated examples of what could make the previously discussed mental models we’re all
familiar with very obscure and hard to understand:

⚫ An email inbox shows – in addition to emails – the names of recently changed documents as entries
(intermingling of different unrelated types of objects).
⚫ Email attachments are associated with a recipient and not with the message itself. Every recipient
receives different attachments (violation of a common mental model).

⚫ Within an online order, items are hidden and seem to “disappear” once they’ve been shipped while other,
unshipped ones remain visible. The user has to go to a different page to find shipped items under “order
history” (obscure business rule).
⚫ An online order can be associated with multiple customers OR an order line (item) can be associated
with multiple separate orders (illogical information architecture).
⚫ On an e-commerce site, item prices aren’t shown and only exposed – as combined order total – only
once a credit card has been entered (break of common mental model from a structure and process
perspective).

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

⚫ Emails from the same sender are considered inter-dependent and cannot be filed in different folders (obscure
business rule and system constraint; a data relationship exists that is not visualized and made apparent to the
user).
While a little far-fetched, hopefully the above examples illustrate how deviation from a common mental model and
other design “sins” can make a user scratch their head when trying to use an application that behaves quite
differently than expected.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

Perception and Sensation

Perception and sensation are two fundamental concepts in human-computer interaction (HCI) that play a
crucial role in how users interact with and experience technology.
Sensation refers to the raw, unprocessed data that our sensory organs receive from the environment. It is the initial
step in the perceptual process, involving the activation of receptors in our eyes, ears, nose, tongue, and skin. For
example, when we see a red apple, the light reflected from the apple's surface is detected by photoreceptors in our
eyes, triggering a cascade of neural signals that ultimately reach the visual cortex in our brains.
Perception, on the other hand, is the process of interpreting and organizing
sensory information to create meaningful experiences. It involves not only the
raw sensory data but also our prior knowledge, expectations, and attention.
When we perceive the red apple, we are not just aware of the light it reflects;
we also recognize it as an apple, understand its color, and perhaps even
anticipate its taste

Key aspects of perception in HCI:

1. Attention: Understanding how users focus their attention on specific elements of an interface is crucial
for guiding their interaction and ensuring that important information is not overlooked.

2. Visual perception: Human vision is highly complex, with various factors influencing how we perceive
visual information, such as color, contrast, shape, and motion. Designers need to consider these factors
to create visually appealing and easy-to-understand interfaces.

3. Auditory perception: Sound can play a significant role in HCI, providing cues, warnings, and feedback.
Designers need to understand how auditory elements are perceived, including pitch, volume, and timbre.

4. Haptic perception: Haptic feedback, such as vibrations or pressure changes, can enhance user
interaction and provide real-time feedback. Designers need to consider the appropriate use of haptic
feedback to avoid sensory overload or discomfort.

5. Cultural and individual differences in perception: Perception can vary across cultures and
individuals, influenced by factors such as age, experience, and cognitive abilities. Designers should
consider this diversity when creating interfaces that cater to a broad range of users.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

Gestalt Principles
The Gestalt principles describe how people perceive visual information as a whole, rather than as
individual parts.These principles include proximity, similarity, closure, and continuity. Gestalt principles are a
set of rules that describe how humans organize visual elements into meaningful wholes. These principles
were developed by Gestalt psychologists in the early 20th century and are still used today by designers,
artists, and psychologists.

The Gestalt principles are as follows:

• Proximity: Elements that are close together are perceived as belonging together.
• Similarity: Elements that are similar in appearance are perceived as belonging together.
• Continuity: Elements that are aligned or connected are perceived as belonging together.
• Closure: We tend to perceive incomplete or partially occluded objects as complete.
• Figure-ground: We tend to perceive objects as being in the foreground and the rest of the scene as being
in the background.
• Prägnanz: We tend to perceive objects in the simplest way possible.

These principles are not hard and fast rules, and there are many exceptions. However, they are a helpful
guide for understanding how humans perceive visual information.

Here are some examples of how Gestalt principles are used in design

• Proximity: A designer might use proximity to group related items together on a web page, such as a
product name and its description.
• Similarity: A designer might use similarity to create a cohesive brand identity, such as using a consistent
color palette and font style across all of the company's marketing materials.
• Continuity: A designer might use continuity to create a sense of flow in a design, such as using a line or
pattern to guide the eye through the page.
• Closure: A designer might use closure to create a memorable logo, such as using a negative space to
create a recognizable shape.
• Figure-ground: A designer might use figure-ground to create a sense of depth in a design, such as using
a light color for the background and a dark color for the foreground.
• Prägnanz: A designer might use prägnanz to create a simple and easy-to-understand design, such as
using a minimal amount of elements and clear typography.

Let’s remember:
Mental models are critical when it comes to making our applications intuitive. Applications whose
underlying mental model is either unclear, illogical or significantly different than common models are
perceived as hard to use and may not be successful. In most cases it will be best to use or build upon existing
mental models. This may include leaning on the physical world and using known metaphors. If we end up
needing to create new mental models, they should be clear, transparent and easy to grasp.
Application design is an art, an art we can’t afford to be bad at. Our users, be they consumers or business
people, are used to high-quality user experiences perpetuated by the mobile revolution. If they don’t “get it”,
they are fast to abandon an application they don’t like or understand. This sets the bar very high not only for
consumer-facing applications, but also for B2B applications. We cannot afford to do this poorly. One of the
key ways to do it well is via use of clear and intuitive mental models.

Prepared by: Jeffrey C. Niedo

 Module 2: The Cognitive Psychology and Perception of HCI

References:
E-Book:
• Rogers, Y., Sharp H., & Preece, J.(2023) Interaction Design: Beyond Human-Computer Interaction
6th Edition. Wiley
• Dix, A. et al (2023) Human-Computer Interaction 3rd Edition. Pearson
• Ardito, C. et. al. (2021) Human-Computer Interaction – INTERACT 2021. Springer
• Johnson, J (2020) Designing with the Mind in Mind: Simple Guide to Understanding User Interface
Design Guidelines 3rd Edition. Morgan Kaufmann
• Becker, C. R.(2020) Learn Human-Computer Interaction. Packt Publishing

Internet:
• What is Human Computer Interaction. (n.d.). Interaction Design Foundation.
https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-
interaction-2nd-ed/human-computer-interaction-brief-intro
• https://agileapplied.com/2017/11/18/mental-models/
• https://xinyicui.medium.com/ui-ux-strategy-a-guide-to-paper-prototype-sketching-3c475ce560f9
• https://www.justinmind.com/prototyping/paper-prototype

Prepared by: Jeffrey C. Niedo