January 13th, 2005

Entrance Auditorium
Umeå Institute of Design - Umeå University
Sweden

Proceedings
 Program & Content

09.30 – 10.00 Registration and Coffee Paper Session II

13.00 – 13.20 Hyoung Won:

Opening Session Designing a Tangible User
Interface for a Refrigerator
10.00 – 10.10 Professor Mike Stott: for a Future Market
Conference Introduction ................................................................... page 29

10.10 – 10.40 Invited speaker Daniel Fällman: 13.20 – 13.40 Patricia Panqueva & David Werjefelt:
From Freedom to Involvement: Sketching relations, an interaction
On the Rhetoric of Mobility in design perspective from research
HCI Research to design
..................................................................... page 1 ................................................................... page 33

13.40 – 14.00 Low Ko Wee:

Paper Session I Enhancing the User Experience
of using an Electric Iron
10.40 – 11.00 Imon Deshmukh: ................................................................... page 37
Using Experience Prototyping as an
Advertising Tool 14.00 – 14.20 Santiago Barriga & Lisa Hultgren:
................................................................... page 11 Information Graphics as an active tool
during the Interaction Design Process
11.00 – 11.20 Mine Danisman Tasar & Malin Mäki: ................................................................... page 42
Capturing the Big Picture:
a Design Case Study with an
Ethnographic Approach 14.20 – 14.50 Coffee
................................................................... page 15

11.20 – 11.40 Paula Guntaur & Kristofer Öberg: Paper Session III
“Experience Portfolio” -
Using Interaction Tools to Explain the 14.50 – 15.10 Linda Bogren & Madlene Lindström:
Concept of Interaction Design Interaction Designers Collaborating in
................................................................... page 19 Matching the Physical and Virtual
Interface in Libraries
11.40 – 12.00 Jessica Chang & Gabriel Åberg: ................................................................... page 46
Applying Cognitive Science Research in
Graphical User Inteface (GUI) 15.10 – 15.30 German León & Ida Ristner:
................................................................... page 23 Using Phonological Strings as a
metaphor to connect Auditive,
Graphical and Tactile User Interfaces
12.00 – 13.00 Lunch and Demo Session ................................................................... page 50

Closing Session

15.30 – 15.50 Niklas Andersson:

Evaluation and closing of the 2005
Interaction Design Student
Conference
Umeå Institute of Design
Designhögskolan i Umeå
From Freedom to Involvement: On the Rhetoric of Mobility
in HCI Research
Daniel Fallman
Umeå Institute of Design
Umeå University, Sweden
Daniel.fallman@dh.umu.se

process, the way mobile information technology is talked

about— its rhetoric—naturally comes to influence the way
ABSTRACT
The concept of ‘mobility’ as it is conceptualized in mobile in which it is perceived; what we believe it to be. In order to
HCI is scrutinized in this paper. The currently applied find new ways of designing mobile information technology,
understanding is often limited to perceiving mobility as it is hence necessary to be able to go outside of its current
corporeal and in spatial and temporal terms exclusively. rhetoric, for which it in turn is crucial to first establish and
While some have attempted to include contextual and social review what the current rhetoric is.
dimensions, their ways of approaching this issue seems Intending to explore what the concept of mobility implies
problematic and in fact only continue a far-reaching by drawing on previous research in mobile HCI is however
separation between the physical and what is seen as the an undertaking anything but straightforward.
social or the subjective. These should however not be seen
as disparate but rather as co-creators of what one perceives Because it as a field of research is still immature, and there
as ‘reality’. are yet comparably few canonized works, the question can
yield a host of different answers according to what
The concept of involvement from phenomenology is community and in what circumstance one chooses to pose
introduced to discuss the possibilities of changing contexts it. Within the small but rather diverse research community
to which use of mobile information technology gives rise. with an interest in mobile HCI, some would argue that the
To conclude, we argue that mobile HCI needs to be thought concept of mobility above everything else poses a host of
of as designing for involvement in these diverse physio- challenging technical issues. These include the need to use
social contexts, rather than as designing technology with a different system architectures, lower computational
spatial and temporal location. capacity, new standards, more focus on power management,
limited bandwidth, intermittent connectivity, distributed
1. INTRODUCTION database access, platform independency, matters that have
This paper will examine not only the concept of mobility, to do with data validity, new kinds of security threats, and
but also its rhetoric. This focus is because it is believed that issues of scalability and reliability [23, 32, 33, 38, 42, 47,
a few rhetorical figures are currently highly important and 54]. Issues of technological nature, such as those attended
influential in the design of mobile information technology by the authors cited above, are a natural and necessary
and that they as such come to guide research as well as elements of any design discipline; whether it is to do with
practice. The aim of this paper is hence to increase our building motherboards for computers; bridges between
understanding of what mobile information technology ‘is’. neighborhoods; coffeepots for air stewardesses; or buildings
The argument is that in our current understanding, a few for those tied to their wheelchairs. What efforts of research
rhetorical figures seem to take on an important role, but that which deal solely with specific technological issues
these may be misleading in several respects. Using typically do not encompass, however, are well-developed
Merleau-Ponty’s [40] dialectical approach to meaning is to understandings of underlying issues—such as, in this case,
say that mobile information technology becomes both what mobility—and what the effects are on the individual and the
we find it to be, but also what we create it to be. In the latter larger society of their technological innovations. Because of
this, no design discipline can simply regard issues of
technology; it must also deal with people—both as
Permission to make digital or hard copies of all or part of this work for designers and as users of artifacts [8, 18].
Permission
personal ortoclassroom
make digital
useoris hard copies
granted of allfee
without or part of thisthat
provided work for are not
copies
personal
made or or classroomforuse
distributed is granted
profit without fee
or commercial providedand
advantage thatthat
copies are bear
copies not Other researchers related to mobile HCI would thus argue
made or distributed
this notice and theforfull
profit or commercial
citation advantage
on the first page. Toandcopy
that copies bear or
otherwise, that prior to the technical issues are the possibilities for new
this notice and
republish, the full
to post citation on
on servers or the first page. To
to redistribute copy requires
to lists, otherwise, or specific
prior
permission
republish, toand/or
post ona servers
fee. or to redistribute to lists, requires prior permission. kinds of work that the concept of mobility brings into play.
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden In the field of Computer-Supported Collaborative Work
Copyright 2005 Umeå University (CSCW), this conceptualization of the enabling quality of
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden

1
mobility to give rise to new ways of working has tended to The current understanding of the concept of mobility thus
govern the discussion. Some of the important previous pre-assumes that mobility is solely about freeing people
efforts within this area are a several user centered studies from geographical constraints: “Their argument of the
which have primarily focused on collaborative mobile significance of mobility, or nomadicity, is clearly confined
work, of which the most well-known include the work to the corporeal characteristic of humans freed from
carried out by Luff & Heath [39], Whittaker et al [56], and geographical constraints thanks to mobile computing
Bellotti & Bly [6]. In one of the earliest studies in the field technologies and services such as mobile phones and
as such, Whittaker et al studied in detail the work of two personal digital assistants” [31]. To come to terms with this
mobile professionals, of whom one was locally mobile current limitation, some authors suggest the concept of
within an office environment while the other was also mobility would be enriched if it would also include the
mobile in the local metropolitan area. The importance of contextual interactions people establish and the social
local mobility, i.e. the ability to walk between different character of that kind of interaction [25, 31]. According to
rooms or buildings at a local site, in supporting this, some seek to reframe mobility in terms of three
communication and awareness has also been examined in dimensions: spatial mobility, temporal mobility, and
Bellotti & Bly’s ethnographic study of a distributed design contextual mobility [31].
team. Luff & Heath’s studies of staff and management at
the London Underground also emphasize issues of 2.1. Spatial and Temporal Mobility
awareness and communication in relation to mobility, Spatial mobility, the most common way of conceptualizing
highlighting not only the importance of human mobility, mobility, mainly denotes corporeal geographical freedom—
but also the mobility of artifacts. There is also a relatively or the proposed ‘nomadicity’ of current urban life and work
small, but growing, interest in focusing on issues of mobile [12, 32, 33, 51, 57]. Nomadicity stems arguable rather from
interaction and mobile user interfaces—for instance the the pervasiveness of modern transportation technology—
matter of displaying and visualizing information on small enabling people to travel with relative ease—than particular
displays, text input, speech input, ergonomics of use, and advances in information technologies [31]. This new way of
new kinds of interaction modalities [25, 43, 46]. living and working has however been found to be rather
poorly supported by traditional information technologies,
2. DIFFERENT NOTIONS OF MOBILITY which in turn has provided incentives to invent and design
While the research community of mobile HCI —the alleged mobile information technology.
experts in mobility—seems fairly indistinct as to its object
If spatial mobility mainly concerns questions of ‘where,’
of study, this can be partly explained by noting that
temporal mobility answers to questions of ‘when.’ It seems
although its members seem to be concerned with the same
that the shift in society to an increased level of mobility—of
issue, mobility, they are primarily pursuing its influence on
both people and information— makes it both troublesome
various areas of interest. While the influence of mobility
as well as less crucial to arrange one’s work or life in a
has been the focus so far, an equal interest in a systematic
linear temporal dimension [31]. Their arguments are similar
unveiling of the notion of mobility is yet to be found. While
to those of Perry et al: “As with notions of space assumed
not often the focus, the notion of mobility— i.e. what one
in the rhetoric of ‘anywhere,’ the notion of ‘anytime’ often
thinks mobility ‘is’—nevertheless has influence on the
assumes a linear notion of time, as opposed to the ‘anytime’
ways system architectures and software applications need to
characterized by the social norms and properties of time
be designed; on the way work can be carried out and
that affect information access and communication behavior.
supported; and on the ways in which users interact with
For example, many people might consider it inappropriate
their computers. But in this focus on what mobility causes,
to make a phone call about work-related issues outside a
the interest in and understanding of what mobility ‘is’ seem
mutually agreed understanding of ‘work time’” [45, p. 326].
to have been at best taken for granted—and at worst
forsaken. Several of these temporal issues have been previously
raised and discussed within HCI [29, 30], also specifically
The view that the concept of mobility itself has not received
so in the area of mobile interactive systems design [29, 45].
proper interest and that most mobile HCI research efforts
But in comparison with other dimensions, such as spatiality,
instead have taken on the character of putting out the fires
the temporal dimension of interactive systems has however
started by mobility, metaphorically speaking, is however
been largely ignored. Nevertheless, one immediate concern
not entirely new [31, 25, 45]. In discussing the perceived
of temporality that has been found to have an immediate
importance of the concept of mobility in our contemporary
impact on interaction and interface issues arise from the
world—visible in for instance the pervasiveness of modern
nature of wireless communications are network delays and
transportation technologies—and how it has reshaped the
outages, leading to unpredictable and unreliable temporal
ways in which people live and work, some have stated that:
characteristics at the user interface [10, 12]. The alleged
“[In] spite of the upsurge of concern with mobility in out
importance of spatiality and temporality in an
social lives, current research perspectives define the notion
understanding of mobility may be further approached and
of mobility quite narrowly, exclusively in terms of humans’
justified by examining the most influential and guiding
independency from geographical constraints” [31].

2
rhetorical figures that surrounds the design and simultaneously writing on the same paper using a
conceptualization of mobile information technology. In the collaborative computer support tool, they would be
next section, one of these figures will be introduced and dependent on time while independent of location. Finally, if
analyzed, as it seems to provide particular insight into some they were writing different parts of the paper at any given
important aspects of the spatial and temporal qualities of location, while every now and then sending each other bits
mobility. and pieces of it to one another by for instance e-mail, they
could be said to cooperate independently of both time and
2.1.1. Rhetoric of ‘Anytime, Anywhere’ place.
Arguable, one of the most widespread rhetorical figures that
persist in the conceptualization of mobility, in research as The latter of these examples, where the two users are
well as in more practice-oriented literature, is that of independent of time as well as place, is clearly most similar
‘anytime, anywhere.’ This rhetorical figure seems to to what is argued in the vision of ‘anytime, anywhere’.
capture the spatial and temporal dimension of mobility. One Nevertheless, as with applications of CSCW, this is by no
of its chief proponents, Kleinrock [see 32, 33], has argued means the only way of in which the phenomenon of
that: “The combination of portable computing with portable mobility can be understood. Wiberg & Ljungberg’s [57]
communications is changing the way we think about study of mobile telecommunication engineers shows that
information processing … We now recognize that access to the mobile service workers they studied were actually often
computing and communications is necessary not only from dependent on both time and location to be able to carry out
one’s ‘home base’, but also while one is in transit and/or their work, and some practical limitations of having
when one reaches one’s destination. Indeed, anytime, ‘anytime, anywhere’ as a guiding vision for the design of
anywhere access” (33, p. 351) mobile information technology thus come into view. Travel
cannot always be avoided, since workers have to be
His claim is that advances in technology—primarily the physically present at certain locations for some tasks—for
amalgamation of information and communication instance attending to telephone poles, customer buildings,
technologies, the miniaturization and the steady network routers, and locations where new cables needs to
improvements in processing power, the growth of the be drawn. Neither are the workers independent of time, as
Internet, distributed computing, global positioning systems, certain tasks were tied to specific time frames—rebooting
and so on—will help realize the vision of access to the telephone network had to be done during night time and
information and services ‘anytime’ and ’anywhere’. At the customer service had to be provided out within 24 hours of
heart of this rhetorical figure is the vision that people reporting [57]. In other words, some practical limitations of
should be able to carry out their computational or the nature of their work make it impossible for the mobile
communicatory activities independently of the hour of the telecommunication engineers to conduct work ‘anytime,
day—i.e. anytime—as well as doing so not being confined anywhere.’
to a specific spatial location—i.e. anywhere. Hence, the
goal of mobile HCI, according to this vision, seems In a similar fashion, Perry et al [45] argue that the notion of
primarily to be one of technological mediation. Mobile ‘anytime, anywhere’ seems to be one of the major premises
information technology should strive to allow the user to be of mobile technology, in that it promises to remove the
connected to one or many networks at all times, to be ready bonds between a person’s location in space and that
at all times to provide the user with appropriate information person’s information and communication resources.
and services. However, they see the notion primarily as a rhetorical
device which contributes—and in some ways even
The appropriateness of ‘anytime, anywhere’ to guide founds—a common discourse within the mobile technology
research and development in mobile HCI has however not industry from which narratives of mobility are produced [7,
remained unquestioned. Wiberg & Ljungberg [57] notes the 45]: “These narratives embody a set of simplistic
similarity between the vision of ‘anytime, anywhere’ and a assumptions about the nature of mobile work. In the
simple model frequently applied in the field of Computer absence of a real understanding of what constitutes mobile
Supported Cooperative Work (CSCW) to denote different work, these narratives are the only fallback in justifying and
kinds of computer supported collaboration applications and shaping design” [45, p. 325].
their spatial and temporal aspects [e.g. 24, 48]. This model
suggests that collaboration supported by computers may As primarily a rhetorical figure for promoting and selling
take place dependently or independently of time as well as mobile technology, the ‘anytime, anywhere’ construct
dependently or independently of location. Two persons seems to serve an important function, while “in terms of
sessioning in front of a single computer jointly composing a understanding technology use in mobile work and
paper is dependent on both space and time, according to informing design through this understanding, such a
this model. Second, if the two were taking split shifts, construct may not be quite so useful, since it misrepresents
sitting in for each other in writing the paper, they could be the reality of the difficulties faced by mobile workers” [45,
said to still be dependent on space but independent of time. p. 324—325].
Third, if they each from a specific location were

3
From these studies, it becomes clear that a particular forms of interaction” [14, p. 88]. In addition, it is
conceptualization only concerned with mobility as well established that Virtual Reality and some Internet
geographical independence may completely neglect or technologies, including online communities, the World
overlook its temporal dimension. Reviewing the notion of Wide Web, and online multiplayer computer games, create
‘anytime, anywhere’ has shown that one must be careful not virtual spaces for their users. In some of these online
to consider the dimensions of spatial and temporal mobility communities, spatial notions are used to steer and guide
as separate, since they are most often highly related and interaction, for instance by requiring two users who want to
influence each other in intricate ways. interact with each other to move their avatars in the virtual
space so that they stand close to and face each other [3, 5, 8,
2.2. Contextual and Social Mobility 50]. In some online communities for instance, spatial
Despite the alleged importance of situated action and the notions are used to steer interaction by requiring two users
social context of use promoted by the CSCW community, who want to interact with each other to move their avatars
one of quite few attempts at providing a framework for in the virtual space so that they stand close to and face each
thinking about mobile use of information technology based other.
on context rather than on detached notions of time and
space is the model of the different types of mobility 2.2.1. Rhetoric of ‘Spaces and Places’
discussed by Kristoffersen & Ljungberg [35]. Interested In spite of the pervasive use of different kinds of spaces in
primarily in issues of neither the exact location of the user interactive systems design, Harrison & Dourish [13, 14, 25]
nor the time of the day, their framework rather focuses on argue that while the notion of space is still important it is
properties of the place in which mobile users, because of typically not the most important organizing factor when it
their corporeal mobility, find themselves situated—as well comes to interaction. Partly based in sociology, they
as the different requirements different kinds of mobilities propose making a distinction between those interactive
have on mobile information technology. They suggest we phenomena which are consequent of the nature of the space
acknowledge at least three kinds of modalities of in which they occur and those that stem from the inhabiting
mobility—traveling, visiting, and, wandering—each with of a certain place [25]. Space, in Harrison & Dourish’s
its own impacts and requirements for both users and sense, seems for the most part be concerned with the
environments. physical—including metaphorically physical— properties
of a given location, an understanding of the physical
Approaching context from a somewhat different
configuration of people and artifacts in this particular
perspective, Harrison & Dourish [13, 14, 25], among a few
setting. As they argue: “Spaces provide physical constraints
others, have argued that the notion of space brought to day
and affordances, based on things like the fact that it is easier
by considering the dimensions of spatiality and temporality
to go downhill than up, that people cannot walk through
typically constitutes a fundamental aspect of also how we
walls, and that light passes through glass” [25, p. 26].
design and think about modern technologies. Not only in
terms of mobile information technology but in effect for all If spaces are configured in different ways, different kinds of
sorts of interactive systems: “Systems designers create behaviors can be supported as well as expected. A small
spaces of all sorts; virtual ones such as ‘name spaces,’ and meeting room with comfortable chairs and a round table
real ones such as the two-dimensional computer ‘desktop’ may support a particular kind of conversation which can be
on which files and icons are arrayed. Across these different expected to be very different from the kind of activity
sorts of spaces, there are certain common elements. For supported by a large auditorium.
instance, things generally appear within the space. There
But the concept of space alone—focusing on physical
can generally be only one object at any given point in
properties—is not enough to grasp the different kinds of
space. Things tend to stay where they’ve been put. Spaces
behavior that appear in different contexts, according to
define distances; things can be nearby or far apart once
Dourish: “Two settings with the same physical
they’re in the space” [14, p. 88].
configurations and arrangements of artifacts may engender
The use of space as an organizing principle in interactive quite different sorts of interactions due to the social
systems design is not a random pick. Drawing on the work meaning with which they are invested. For example,
on metaphors carried out by Lakoff & Johnson [36] is to although the stage of an academic conference is physically
suggest that spatial notions of distance and direction are configured in ways very similar to a concert hall, it is
basic and important aspects of human cognition, and as generally not appropriate to get up and sing there” [14, p.
such a potentially useful resource for interactive systems 89].
design. In CSCW, the notion of space as an organizing
Rather than being guided only by the physical properties of
principle has in effect been particularly influential [14]. For
the settings in which we are, our behavior is as much
instance, a ‘shared workspace’ has become a common and
guided by social norms and agreements. While ‘space’
expected feature of many collaborational tools, which use
refers to the context’s physical organization and
“space as a way for people to manage their accessibility,
characteristics, ‘place’—according to Harrison &
orient toward shared artifacts, and provide a ‘setting’ for
Dourish—rather refers to the way in which we are framed

4
by social conventions to behave accordingly. Hence places, contexts in at least two ways [51]. First, change in
contrary to spaces, provide an “appropriate behavioural involvement can take place because of corporeal mobility
framing; […] engender a set of patterned social responses” of people. This suggests that the mobile human being—by
[13, p. 25—26]. virtue of some level of freedom in terms of spatial and
temporal location, typically amplified by modern
3. FROM SPACES TO PLACES (AND BACK?) transportation technologies—can physically appear in very
But is space then different from place? Waterworth [53] different contexts, even on the same day. In previous
agrees with Harrison & Dourish in that first, socializing in societies with less well-developed transportation
places entangles the individual in conventions of what is technologies, the possibilities of rapidly changing contexts
socially acceptable and thus appropriate behavior, and were considerably more limited than in today’s world. It is
second, that all social places do not need to be spatial. But however obvious that social boundaries between people of
on the contrary to them, Waterworth argues that: “[A] different social classes, ethnical or cultural backgrounds, or
‘place’ in normal usage does not of itself imply what they which simply have different educations or interests, still
mean by ‘placeness.’ A place can simply be a location. To exists and still greatly impedes people’s possibilities of
avoid this confusion, we need to be explicit about whether being mobile. Today, with the ease of transportation and
we mean by ‘place’ a particular location, or a social communication, it might even be that social and cultural
occasion to which a set of conventions applies. […] To use issues of mobility are the most difficult issues to overcome.
the term ‘place’ to mean the latter seems to me confusing, Second, current technologies such as modern transportation
since we are really talking about appropriate behaviour for systems and not least various kinds of information
a social occasion, wherever it takes place. The expression technologies also provide the means for humans to be able
‘behaving out of place’ refers metaphorically to a time to appear in a multitude of social context, sometimes
when different social occasions took place in different simultaneously, without needing to appear there physically.
places (in the sense of location in space)” [53, p. 135]. Meyrowitz suggests that “Communication and travel were
once synonymous. Our country’s communication channels
With this, it is suggested that two similarly configured
were once roads, waterways, and railroads. Communication
spaces, such as Dourish’s concert hall mentioned in a
speed was limited to the speed of human travel. […] With
previous passing, do not result in different behaviors
the invention and use of the telegraph, the informational
because of place but rather because they are simply
differences between different places began to erode. […]
different kinds of social occasions. Thus, according to
Physical distance as a social barrier began to be bypassed
Waterworth, a concert hall as a place facilitates singing as
through the shortening of communication ‘distance.’ […]
well as conferences, they occur in the same place as well as
Movement from situation to situation and from social status
in the same space, but it is the social occasion that
to social status once involved movement from place to
determines what the appropriate behavior is. In this way, it
place. A place defined a distinct situation because its
seems that the metaphorical use of place that Harrison &
boundaries limited perception and interaction. Like all
Dourish advocate—in expressions such as ‘behaving out of
electronic media, the telegraph not only defies limits
place’—may easily be confused with its literal meaning—
formerly set by distance, but also bypasses the social rite of
i.e. ‘your place or mine?’
‘passage,’ that is, the act of moving both physically and
The issue of how social conventions govern our behavior socially from one ‘position’ to another” [41, p. 116]. Hence,
and the change new kinds of media and information numerous recent technological advances in what is
technologies has brought by is explicitly and thoroughly sometimes referred to as Computer-mediated
tackled by Meyrowitz’s ostensibly ignored No Sense of Communication (CMC), such as the previously mentioned
Place [41]. In this work, Meyrowitz discusses electronic Internet technologies—e.g. e-mailing, instant messaging,
media in relation to spaces and places: “Before electronic discussion boards, mailing lists, multi-user dungeons
media, there was ample reason to overlook the difference (MUDS)—as well as technologies such as cellular phones
between physical places and social situations. […] A given and video-conferencing, help people communicate despite
place-situation was spatially and temporally removed from of being physically dispersed and they may accordingly
other place-situations. It took time to travel from situation never meet physically [50].
to situation, and disturbance was a measure of social
Similar to the issue of spatiality and temporality, the
insulation and isolation. Since rooms and buildings can be
importance of context to interactive systems is not unique
entered only though set doorways, people once could be
to mobile devices. It has already been reflected in various
included in and excluded from situations in clearly
ways in interactive systems research related to HCI. Some
observable and predictable ways. Electronic media,
of the fields which have been especially concerned with
however, make significant inroads into the situations once
contextuality are ubiquitous and pervasive computing [52,
defined by physical location” [41, p. 116].
54]; tangible computing [14, 28]; social computing [14, 16,
‘Modern technologies’—inclusive of information 22]; wearable computing [1, 34, 43]; augmented reality [2,
technology—in this way seem to influence the way in 21]; and obviously the previous efforts in Artificial
which people can change their involvements in different Intelligence [15, 16], which however seems often neglected

5
in these fields. Several attempts have been made to people and circumstances; different ways of doing and
prototype context-sensitive devices which hold some kind thinking; different cultures, codes, and moods; different use
of awareness of their location, their physical context, other of language and degree of mutual recognition, and so on, in
devices in the spatial and temporal vicinity, and whom their which and with which the human being—contrary to the
user is [9, 10, 52], e.g. the Xerox PARCTab [52], a system car—gets involved. Thus, mobility for a human being is not
aware of its location from which some simple but allegedly only a matter of spatiality and temporality—but even more
valuable services are provided. While the focus of these important is that mobility is a matter of shifting contexts; of
efforts so far has been that of designing specific devices and changing involvements. The suggested importance of
applications, a more theoretical understanding of the role of mobility as involvement is further supported by and in line
context—and thus involvement—in mobility has however with some previous research carried out into issues of
not been presented [12]. mobility and collaboration, where it has been established
that one of the main reasons for people to want to be mobile
3.1. Being-in-the-world is to have face-to-face meetings with others [6, 37, 45].
From the proposed distinction between spaces and places, Thus, rather than striving to be independent of time and
one is easily led to believe that there is a disparity—hence a location, people on the contrary become mobile in different
kind of parallel co-existence— between the two constructs, ways—not only corporeal—to be able to get involved in
between two different ‘worlds’. In one of these worlds, only different physical and social contexts.
‘physical’ things seem to count; things have locations in
time and space, weight, and shape, but nothing more. In the 4. ASPECTS OF FREEDOM VERSUS INVOLVEMENT
other world, seen as the social or ‘subjective’ world, human Some further implications from perceiving the world as a
volitions are supposed to roam. But in light of the simultaneously spatial, temporal, and social setting for
phenomenological attitude which pervades this work, it human action have been provided by Merleau-Ponty [40],
becomes both possible and neccesary to question this view and specifically from recalling his notion of the human
of the world, which sees the world as largely twofold. being as a body-subject. As noted, the view of space as
Hence, to unveil any dimensions of mobility, the different from place promotes a view of the world where
relationship between these two proposed ‘worlds’ must thus one sees the human as concurrently being part of two quite
be further analyzed. To begin this examination, and different worlds; first, a physical world—an ‘objective’
instantaneously relate it to some previous findings in HCI, world, in which humans are seen as any other object—and
it is obvious that on a basic level Harrison & Dourish’s [13, second, a social world—a ‘subjective’ world, created by
14, 25] argument has some relation to Suchman’s each and every individual. Merleau-Ponty took off from
Heidegger’s [27] argument that human being-in-the-world
[49] finding that human action is a largely situated
must because of Dasein’s inhabiting character be
activity—that the specific context or circumstance in which
understood as a question of involvement over inclusion, and
humans are situated are of the highest importance for
that Dasein’s being-in must accordingly be distinguished
understanding human action. For our purposes, this may be
from an object’s being in as the first by way of being is
translated into a suggestion that human beings are not only
involved with things while the latter’s being is indifferently
mobile in spatial and temporal terms, but that they are also
‘occurent’. But, which is Heidegger’s point, there is a
so in terms of their involvements in contexts.
human tendency to interpret ourselves in terms of the
One of Heidegger’s [27] most valuable claims, and one to objects with which we deal, and in this conceals our active
which this work returns many times, was his assertion that involvement with our world. Merleau-Ponty too holds that
one must not understand a human being’s (Dasein) human subjects actively inhabit the world. But if we inhabit
existence (Being-in-the-world) as simply a matter of spatial the world in the way suggested by Heidegger and Merleau-
and temporal location with respect to other objects. Ponty, it cannot easily be seen as distinct from us as
Dasein’s being-in-the-world is fundamentally different perceivers, and likewise we as perceivers cannot see
from the being in the world of objects such as trees, stones, ourselves as distinct from the world. This character of
and bridges. In other words, human mobility must be involvement in the world made Merleau-Ponty conclude
thought of as different by nature from the mobility of a car. that bodysubjects do neither solely impose meanings on the
But in what way is it different? While the car could also be world, nor do they simply find suitable structures of
said to be mobile in terms of the spatial and temporal meaning in the world, but rather that body-subjects through
dimensions—one travels with the car from home to work, interplay with the world both find as well as create
leaves it in a parking lot throughout the day, and then takes meaning. Hence, according to Merleau-Ponty, to inhabit is
it back at night—its mobility is different from the mobility to create meaning and structure as a result of a dialectical
of a human being in that the car is not involved in the world relationship in which the human subject and the world are
in the same way as is a human being. In some sense, this fundamentally intertwined and come to structure each other.
difference seems to be precisely what Harrison & Dourish In Dourish’s [15] example of the concert hall that is being
are arguing with their proposed distinction between spaces used for both purposes of singing and conferencing, this
and places. Different contexts involve different sets of would be to say that whether or not a specific event is a

6
concert or a conference is neither something one simply understanding the corporeal mobility of human beings and
finds in the world (i.e. that it is not simply a quality of that artifacts. But mobility cannot and should not only be
place in itself); nor is it something solely created in the understood as a matter of location in time and space.
minds of those involved (i.e. without physical and structural Perhaps more important is understanding mobility’s way of
elements). It is rather the interplay between a shared altering the user’s context; of allowing the user to involve
mindset of people situated in an appropriate setting which in different contexts. As the latter parts of this paper have
allows the mindset to come about and nurtures it along the shown, a ‘context’ is by definition both physical and social,
way. The event of either a concert or a conference is thus and is so at the same time. Capturing one’s physio-social
necessarily based in both physical and social elements, context computationally for providing input into interactive
simultaneously and inseparably. systems is however a challenging, if not an impossible,
undertaking. As the Cartesian way of understanding space
Hence, social occasions, such as conferences or concerts
is much easier to represent and deal with in a computer
which call for appropriate behavior according to cultural
application, the shift to a focus on the physio-social context
conventions, cannot be seen as detached from either what
in which a human being is situated and involved rather
Harrison & Dourish [25] call space or place. The problem
directs attention to questions of meaning, at which
with the argument of a conference or a concert as simply a
computational systems are notorious [15, 16].
social occasion, drawing on Merleau-Ponty, is that the
world is then thought of as external to the minds of the Any notion of mobility applied in mobile HCI must because
conference attendees holding the idea of the conference of the physio-social nature of context not adopt the naïve
event, something which brings back Cartesianism in terms Cartesian model where the world is solely thought of in
of the human subject—the thinking subject—as detached temporal and spatial terms. While these are the dimensions
from the world—the body. The conference is not purely a of mobility which are highlighted by the pervasive
subjective phenomenon; its taking place also necessarily rhetorical figure of ‘anytime, anywhere’, it is important not
needs to be founded in features of the occasion itself which to conceal the situatedness and involvement of human
allows it to be found as a conference by human subjects, action, in which the physical and social setting, the context,
some of which are physical or structural features. A of human action is of highest importance. The importance
conference cannot generally take place in certain physical of context in mobile HCI would resemble in many ways
settings, e.g. a duck pond or at the local fishmonger’s— phenomenology’s interest in the life-world; the world as
even if the attendees try to persuade themselves and act as experienced by someone being in a specific setting. As the
if it could. This is simply because a conference is a life-world is made up of a complex system of meaning
conference both because one sees and thinks of it as a patterns, some which transcend individuals and form social
conference as well as the event itself through embodying patterns, it is intrinsically difficult to grasp for any
certain physical and social properties allows itself to be computational system [15, 16].
seen as a conference. Thus, conferencing in the duck pond
A conclusion of this paper must thus be that, first, mobile
will be carried out by the involved as a conference-in-a-
HCI needs to be concerned with three elements of
duck-pond—unlike a conference carried out in a setting
interaction; human, computer, and world. But second, while
which allows itself to be seen as a place for conferencing.
that these form a relationship in which all are of interest for
Some of these latter properties include social elements,
mobile HCI, it is a relationship which must be
such as people acting and behaving in certain ways, as well
conceptualized and treated as non-neutral because of the
as necessary physical elements, people sitting in certain
computational difficulties involved in capturing context;
ways, chairs and tables organized in accordingly, and
appropriated rooms and buildings. In this way, the body- i.e. forming any kind of computational understanding of
subject and the world—of which the latter must be world. Context-awareness has typically been pursued
understood as both physical and social—are highly through designs which try to capture and interpret what
intertwined, and it becomes even more clear why one goes on in the users’ close physical vicinity, intending to
cannot be understood without the other. provide them with appropriate information or services
accordingly. While it has been noted that this is a highly
challenging and difficult trait to follow, there is also the
possibility of designing not primarily for the system to be
5. MOBILITY AS INTERPLAY BETWEEN FREEDOM AND
INVOLVEMENT able to interpret the user’s context, but rather design the
In this paper, it has been suggested that the context in which computational systems in such a way that they do not
humans are founded, the immediate physical contexts as interfere with their users’ contextual activities. This is to
well as the various social contexts in which they say, computational devices which do not intend to grasp
simultaneously roam plays an as important role for the human involvements, but which have been designed
concept of mobility as do the dimensions of spatiality and according to a basic understanding which suggests that for
temporality, which are comparably well-documented within the situations and circumstances in which mobile devices
mobile HCI. The spatial and temporal dimensions are two become used, the context constitutes an important element
important dimensions of mobility, useful especially for and determines both what is wanted as well as what the user

7
finds appropriate. This could be to argue for a design REFERENCES
philosophy of mobile HCI in which devices are aware of 1. Abowd, G. D., Dey, A. K., Brotherton, J., Orr, R. J.:
their users’ being-in—physical and social involvements— Context Awareness in Wearable and Ubiquitous
which sometimes make certain behavior of the system Computing. First International Symposium on Wearable
inappropriate and unwanted. Computers, IEEE Computer Society Press, Los
Alamitos CA (1997) 179—180
The most obvious example of this, of course, is the mobile
phone. While the mobile phone’s buzzing interacts with its 2. Aliaga, D. G.: Virtual Objects in the Real World. Com.
user as a notification that someone wants to communicate, of the ACM, 40, 3 (1997) 49—54
it is also likely to disturb and distract other people in the 3. Anders, P.: Envisioning Cyberspace: Designing 3D
vicinity—the context, or world in which this particular Electronic Spaces. McGraw-Hill, New York NY (1999)
instance of interaction takes place. At times, the buzzing of 4. Barley, S. R.: On Technology, Time, and Social Order:
a mobile phone is a useful and direct way of notifying its Technically Induced Change in the Temporal
user, but in certain situations it is just the wrong thing to do. Organization of Radiological Work. In Dubinskas, F.
When listening to a live concert, when giving a keynote at a A.: (Ed.) Making Time: Ethnographies of High-
conference, when gazing in awe at Michelangelo’s Pietà, or Technology Organizations. Temple Univ. Press, Phil PA
when just trying to get some sleep, are all examples of (1988)
when a loud buzzing just is not the right way of notifying
the user. But the mobile phone does not know whether it is 5. Benedikt, M. (Ed.): Cyberspace: First Steps. MIT Press,
at home, at a conference, in Basilica di San Pietro, or at the Cambridge, MA (1991)
local fishmonger’s. Even if the cell phone knew this, it 6. Bellotti, V., Bly, S.: Walking Away from the Desktop
would still now know whether the specific situation would Computer: Distributed Collaboration and Mobility in a
allow it to bleat out ‘Here Comes the Sun’—whether its Product Design Team. In Proc. of Conference on
user is waiting for the phone to ring; sleeping; taking a Computer-Supported Cooperative Work (Boston MA,
guided tour; or in a sensitive argument with the user’s Sept). ACM Press, New York NY (1996) 209—218
fiancée. Similarly, it does not know whether or not its user
7. Churchill, E. F., Wakeford, N.: Framing Mobile
is listening to the keynote, giving the keynote, or cleaning
Collaboration and Mobile Technologies. In Brown, B.,
up the place afterwards. And if it knew that its user is at the
Green, N., and Harper, R. (Eds.) Wireless World: Social
fishmonger’s, it cannot know what the user is doing there,
and Interactional Implications of Wireless Technology.
or even if the user is the fishmonger. While these are all
Springer-Verlag New York NY (2001)
things that the mobile phone does not and cannot know, the
solution is however not as simple as to tell our mobile 8. Coyne, R.: Designing Information Technology in the
information technology devices to start to notice and pay Postmodern Age. MIT Press, Cambridge MA (1995)
attention to their contexts and their user’s involvements. As 9. Davies, N., Blair, G., Cheverst, K., Friday, A.:
have been noted, catching the physio-social context is Supporting Adaptive Services in a Heterogeneous
computationally difficult—understanding context in terms Mobile Environment. In Cabrera, L.-F. &
of human involvements seems computationally impossible Satyanarayanan, M. (Eds.) Proceedings of the Workshop
[15, 16]. But that does not mean that we, like Heidegger on Mobile Computing Systems and Applications (Mobile
fears, must resort to understanding ourselves in light of the ’94, Santa Cruz, CA, Dec.) IEEE Computer Society
objects with which we deal. Press, Los Alamitos CA (1994) 153—157
The obvious significance of the proposed concept of 10. Davies, N., Mitchell, K., Cheverst, K., Blair, G. S.:
involvement for Human—Computer—World Interaction, Developing a Context Sensitive Tourist Guide.
acknowledging the inherent difficulties involved in catching Technical Report, University of Lancaster, Lancaster
and interpreting it computationally, hence promotes a UK (1998)
design philosophy which seeks to recognize its importance 11. Deleuze, G., Guattari, F.: Nomadology. Semiotext, New
but does not necessarily attempt to develop ways of York NY (1986)
understanding it fully, and where it partially attempts to do
so realizes that it can never have a complete understanding 12. Dix, A., Rodden, T., Davies, N., Trevor, J., Friday, A.,
of meaning in a human context. Such a design philosophy Palfreyman, K.: Exploiting Space and Location as a
would promote designs which seek to enhance—or at least Design Framework for Interactive Mobile Systems.
which do not interfere with or literary get in the way of— Transactions on Computer—Human Interaction, Vol. 7,
any possible human interactions that may arise because of No. 3, Sept. (2000) 285—321
involvement in a particular physio-social context, of which 13. Dourish, P.: Where the Footprints Lead: Tracking Down
the system is not and cannot be fully aware. Other Roles for Social Navigation, In Höök, K.,
Benyon, D., Munro, A., (Eds.) Social Navigation of
Information Space, Springer-Verlag, London UK (1999)
15—32

8
14. Dourish, P.: Where the Action Is: The Foundations of GA, March) ACM Press, New York NY (1997) 234—
Embodied Interaction. MIT Press, Cambridge MA 241
(2001) 29. Johnson, C. W.: The Impact of Time and Place on the
15. Dreyfus, H.: What Computers Can’t Do: A Critique of Operation of Mobile Computing Devices. In
Artificial Reason, Freeman, San Francisco CA (1972) Proceedings of the Conference of HCI ’97 on People
16. Dreyfus, H.: What Computers Still Can’t Do, MIT and Computers XII ( HCI ’97, Bristol, UK), Springer
Press, Cambridge MA (1992) Verlag, London UK (1997) 175–190
17. Dryer, D., Eisbach, C., Ark, W.: At What Cost 30. Johnson, C. W.: Representing the Impact of Time on
Pervasive? A Social Computing View of Mobile Human Error and Systems Failure, Interacting with
Computing Systems. IBM Systems Journal, 38, 4. Computers, 11 (1998) 53—86
(1999) 652—676 31. Kakihara, M. & Sørensen, C.: Mobility: An Extended
18. Ehn, P.: Work-oriented Design of Computer Artifacts. Perspective. In Sprague, R. (Ed.) 35th Hawaii
Arbetslivscentrum, Sweden (1988) International Conference on System Sciences, HICCS-
35 (Big Island, Hawaii), IEEE, Los Alamitos CA (2002)
19. Fallman, D.: Wear, Point, and Tilt: Designing Support
for Mobile Service and Maintenance in Industrial 32. Kleinrock, L.: Nomadic Computing: An Opportunity.
Settings, In Proceedings of Designing Interactive ACM Sigcomm, Computer Communication Review,
Systems, DIS2002 (London UK, June 25-28) ACM Press Vol. 25, No. 1 (1995) 36—40
(2002) 33. Kleinrock, L.: Nomadicity: Anytime, Anywhere in a
20. Fallman, D.: An Interface with Weight: Taking Disconnected World. Mobile Networks and
Interaction by Tilt beyond Disembodied Metaphors, In Applications, Vol. 1, No. 4, Jan (1996) 351—357
Paterno, F. (Ed.), Mobile Human-Computer Interaction 34. Kortuem, G., Segall, Z., Bauer, M.: Context-Aware,
Mobile HCI 2002 (Pisa, Italy, Sept) LNCS 2411, Adaptive Wearable Computers as Remote Interfaces to
Springer-Verlag (2002) ‘Intelligent’ Environments¸ Second International
21. Feiner, S., MacIntyre, B., Hollerer, T., Webster, A.: A Symposium on Wearable Computers (Pittsburgh PA,
Touring Machine: Prototyping 3D Mobile Augmented Oct) IEEE Computer Society Press, Los Alamitos CA
Reality Systems for Exploring the Urban Environment, (1998) 58—65
First International Symposium on Wearable Computers, 35. Kristoffersen, S. & Ljungberg, F. Mobile Use of IT. In
IEEE Computer Society Press, Los Alamitos CA (1997) Dahlbom, B., Ljungberg, F., Nuldén, U., Simon, K.,
74—83 Stage, J., & Sørensen, C. (Eds.) Proceedings of IRIS 19.
22. Fitzpatrick, G., Mansfield, T., & Kaplan, S.: Locales (Gothenburg) Gothenburg Studies of Informatics (1999)
Framework: Exploring Foundations for Collaboration 36. Lakoff, G., Johnson, M.: Methaphors We Live By. Univ
Support. In Proc. of OzCHI‘96 (Hamilton, NZ). IEEE of Chicago Press, Chicago (1980)
(1996) 34—41 37. Lamming, M., Eldridge, M., Flynn, M., Jones, C., &
23. Forman, G. H., Zahorjan, J.: The Challenges of Mobile Pendleburry, D.: Satchel: Providing Access to Any
Computing. IEEE Computer, (1994) 38—47 Document, Any Time, Anywhere. ACM Transaction of
24. Grudin, J.: CSCW: Introduction to Special Issue. Computer-Human Interaction, 7, 3 (2000) 322—352
CACM, Vol. 34, No. 12 (1991) 30—34 38. La Porta, T., Sabmani, K. K., Gitlin, R. D. Challenges
25. Harrison S., Dourish, P.: Re-Place-ing Space: The Roles for Nomadic Computing: Mobility Management and
of Place and Space in Collaborative Environments. In Wireless Communications MONET 1, 1. (1996) 3—16
Proceedings of ACM Conference on Computer- 39. Luff, P., Heath, C. Mobility in Collaboration. In
Supported Cooperative Work CSCW’96, (Boston, Proceedings of the Conference on Computer-Supported
November) ACM Press, New York NY (1996) Cooperative Work. ACM Press, New York NY (1998)
26. Harrison, B., Fishkin, K., Gujar, A., Mochon, C., Want, 305—314
R.: Squeeze Me, Hold Me, Tilt Me! An Exploration of 40. Merleau-Ponty, M.: Phenomenology of Perception.
Manipulative User Interfaces. In Proceedings of Human Routledge, London UK (1962)
Factors in Computing Systems (CHI 98), ACM Press, 41. Meyrowitz, J. No Sense of Place: The Impact of
New York NY (1998) 17—24 Electronic Media on Social Behavior. Oxford University
27. Heidegger, M.: Being and Time. Harper and Row, New Press, New York NY (1985)
York NY (1962) 42. Nichols, R. K., Lekkas, P. C.: Wireless Security.
28. Ishii, H., Ullmer, B.: Tangible Bits: Towards Seamless McGraw Hill, New York NY (2002)
Interfaces between People, Bits and Atoms. In 43. Pascoe, J. Adding Generic Contextual Capabilities to
Pemberton, S. (Ed.) Proc.of the ACM Conference on Wearable Computers, Second International Symposium
Human Factors in Computing Systems (CHI97) (Atlanta

9
 on Wearable Computers (Pittsburgh PA, Oct) IEEE 52. Want, R., Schilit, B. N., Adams, N. I., Gold, R.,
Computer Society Press, Los Alamitos CA (1998) 92— Petersen, K., Goldberg, D., Ellis, J. R., Weiser, M.: An
99 Overview of the ParcTab Ubiquitous Computing
44. Pascoe, J., Ryan, N., & Morse D.: Using while Moving: Experiment. IEEE Personal Communications. 2, 6, Dec.
HCI Issues in Fieldwork Environments. In ACM (1995) 28—43.
Transactions on Computer—Human Interaction, Vol 7, 53. Waterworth, J. A. Spaces, Places, Landscapes and
3, (2000) Views: Experiential Design of Shared Information
45. Perry, M., O’Hara, K., Sellen, A., Brown, B., Harper, Spaces. In Munro, A., Höök, K., & Benyon, D. (Eds.)
R.: Dealing with Mobility: Understanding Access Social Navigation of Information Spaces. Springer-
Anytime, Anywhere. ACM Transactions on Computer Verlag, London UK (1999)
Human Interaction (TOCHI), Vol. 8, Issue No. 4, 54. Weiser, M.: The Computer of the 21st Century.
December (2001) 323-347 Scientific American, 265, 3 (1991) 66—75
46. Rekimoto, R.: Tilting operations for small screen 55. Weiser, M.: Some Computer Science Issues in
interfaces, Proceedings of UIST’96, (Seattle WT), ACM Ubiquitous Computing. Communications of the ACM,
Press, New York NY (1996) 167—168 36, 7 (1993) 75—85
47. Satyanarayanan, M.: Fundamental Challenges in Mobile 56. Whittaker, S., Frohlich, D., Daly-Jones, O.: Informal
Computing. Proceedings of ACM SigMobile, 1, 1, Workplace Communication: What Is It Like and How
(1997) 1—7 Might We Support It? In Proceedings Conference on
48. Shneiderman, B.: Designing the User Interface. Human Factors In Computing Systems (CHI94). ACM
Addison-Wesley, New York NY (1997) Press, New York NY (1994) 131—137
49. Suchman, L.: Plans and Situated Actions: The problem 57. Wiberg, M. & Ljungberg, F.: Exploring the Vision of
of Human—Machine Communication. Cambridge Anytime, Anywhere in the Context of Mobile Work. In
University Press, Cambridge UK (1987) Malhotra, Y. (Ed.): Knowledge Management and Virtual
Organizations: Theories, Practices, Technologies and
50. Turkle, S.: Life on the Screen: Identity in the Age of the Methods. Idea Group Publ (2001) 157—169.
Internet. Simon & Schuster, New York NY (1995)
51. Urry, J.: Sociology beyond Societies: Mobilities for the
Twenty-First Century. Routledge, London UK (2000)

10
 Using Experience Prototyping as an Advertising Tool
Imon Deshmukh
Institute of Design, Umeå University
SE-901 87 Umeå, Sweden
umea@imon.org | http://imon.dk
+46 (0)73 836 25 75
ABSTRACT
Experience Prototyping
This paper suggests using the design practice of experience
Experience prototyping is widely defined as any kind of
prototyping as an online marketing tool. While building a
representation, in any medium, that is designed to
prototype of a communication concept based on Magnetic
understand, explore or communicate what it might be like
Poetry, it became apparent to us that technology has
to engage with a product, space or system [1]. Within our
evolved to a point where it has become achievable to
field of interaction design, it is an invaluable aid in
recreate experiences to a sufficient degree with simple
perfecting and enhancing a concept through human
design tools and deliver them to a remote location. We
feedback. With visual and audio simulations so widely
propose carrying over this practice to consumers.
available, it has become easier to recreate a multisensorial
Presenting a virtual representation of a product would let
replication of the real experience. The main point of this
users evaluate important aspects of a product to determine
paper is: Why employ this technique only in the
if they like the experience enough to buy it. Moreover, we
development stage of a product?
believe it would shift the whole concept of advertising
away from a device to sell a product by any means
Evolution of Prototyping Tools
necessary to a more honest and ethical presentation.
In the last decades, multimedia applications and tools have
continuously become more affordable. Presently a GUI
Author Keywords
interface can be simulated at a very high degree of realism
Experience prototyping, marketing, advertising, user testing,
with widely available and affordable tools such as
user evaluation, market research
Macromedia Flash or Director in both of which graphics are
photo realistic and sound is integrated. Of course, real
INTRODUCTION
Advertising has become, or perhaps always has been, an art experience prototyping, the kind used by professional
in deception, and it is generally accepted as such by society. designers, involves several other tools and much more
Though the objective is usually to motivate a consumer to sophisticated technology than computer software. For the
purchase certain goods or services, it is perfectly normal to sake of this paper’s argument, however, simple interface
achieve this goal by interacting with him or her in a way prototyping tools are mostly adequate.
that has virtually nothing to do with the actual qualities of
Experience prototyping as advertising
the merchandise. Instead, advertisers generally rely on
Experience prototyping has proved itself as a highly
creating positive associations with their products and their
effective methodology in designing products and I believe it
brand, often appealing to very primal instincts. Besides
would be equally powerful if we applied this tool outside
questions of ethics, this type of advertising results in the
the first development stages, targeting end consumers on
quality of the product playing a limited role since it is sold
the web. The reasons for this are more than one: views and
on something different than its own merits. A consequence
clicks and other interactions are far easier to track on the
of this is that a large portion of a product’s budget is spent
web than any other media[2], and the audience is
on marketing it instead of developing it into a better product.
comparatively large [2]. With some small incentives given
While this type of advertising can sometimes produce
to web surfers, many aspects of a product can be tested by a
inspired and innovative artworks, it is ultimately a
large group of people and trends and preferences can be
deception of the consumer.
identified[2].
Permission to make digital or hard copies of all or part of this work for Another use is more relevant to this paper and that is online
personal or classroom use is granted without fee provided that copies are not advertising. As stated before, it has become very easy to
made or distributed for profit or commercial advantage and that copies bear
create powerful software simulations on a computer screen.
this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior permission. Obviously, there are limits to what realities a two-
dimensional display can recreate but as more and more
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden products are moving to circuit based interfaces (a sunroof

11
that is opened by curbing a mechanical lever vs. an Saab Automobiles released an e-brochure where they let
electronic button), the possibilities are expanding. prospective customers press some buttons to watch the
functions in action, e.g. opening the top (Figure 1). Having
Despite the clear deficiencies of using computer simulation
actual control and seeing the result of an action makes a
to stimulate the senses compared to a real experience in the
much stronger impression than a text description of all the
physical world, the gap is shrinking continuously. Touch-
great features of the product.
sensitive screens are more widespread and affordable and
force feedback, while still mainly used in video games will
probably soon be put to use in other contexts.
Until now the main obstacle to this idea has been the lack of
technology, but this is no longer the case. These new
affordable technologies represent a great leap toward giving
a full experience of a product remotely. A theoretical
example using the touchscreen is presented later in this
paper.

Benefits
It is practically impossible to precisely predict how this
type of advertising would affect marketing and product
development. That said, here are some (optimistic)
prognoses.
If the marketed product itself becomes the primary message
in its own advertising campaign, more value will be placed
on improving the experience of using the product. Currently
marketing is used to create positive associations with the
product through channels that may be completely unrelated
to using it. This would change because consumers would
come to expect a virtual test drive from certain types of
products (particularly electronics) and withholding that
would arouse suspicion. (“Why don’t they let us actually try
out the cell phone?”) Figure 2. The Sony Ericsson ad on the cover page of the online
The high degree of measurability that is possible in online edition of the New York Times.
ads would allow developers gauge user reactions and When launching the P910a, a new ‘smartphone’, Sony
preferences. Feedback from a pre-release ad campaign, for Ericsson ran an interactive banner inviting the reader to
example, could be used to tweak the product to ensure that pick up a stylus which is shown against a black background
it is optimized at launch. (see Figure 2). When the stylus is clicked on, the banner
Most importantly, it changes the whole idea of advertising expands to reveal the phone.
as a device to seduce consumers into buying a product into
more of an honest try-it-before-you-buy-it process.

EARLY EXAMPLES
There are already instances today that replicate an
experience, albeit to a much more limited degree than what
I believe is possible. Nevertheless they are a suitable
preview of what is feasible and how it could look in the
future.

Figure 3. The banner lets the user interact with the content on
a higher level than most online ads.
Once this happens, the cursor, when it is inside the banner
area, appears as the stylus. The user is invited to use this
stylus to draw on the screen of the phone (Figure 3). To add
Figure 1. Clicking on the buttons on the ignition key lets the more realism, there is an added working feature that allows
user watch some of the functions in action. the user to change the colour of the pen (Figure 4). It then

12
lets the user ‘send’ the drawing in an e-mail. (It does not We used different types of terminals (terminals represent
actually send an e-mail, it merely triggers an animation that edrefrigerators), because it was unfeasible with our
takes eventually leads to another scene where the user can equipment to recreate all the properties of a ‘Poetry in
roll over different areas to find out more details). At the end Motion’-equipped refrigerator in the same terminal. One
there is even an option to print out the sketch the user made. was the ‘Powerwall’, an approximately 2 metre tall and 3
Whether this is a feature in the actual phone is unclear. metre wide structure with a translucent screen that we back
projected an image of a fridge with the magnets on the face.
Ideally the Powerwall would be touch-sensitive and allow
us to move the magnets with our fingers. This was not a
possibility however and we improvised by using eBeam [3],
an interactive whiteboard technology. eBeam allowed us to
use ultrasonic markers to move the magnets around (Figure
6).

Figure 4. The ability to change ink colour is a fully functional

feature.

POETRY IN MOTION
The Concept
Poetry in Motion is a concept conceived by Linda Bogren, a
graduate student at the Institute of Design at Umeå
University, which uses a graphical form of Magnetic Poetry
on refrigerators as a remote communication tool. When a
person in a household moves words on their fridge, another
person in a different household can see the same words
Figure 7. A touch-sensitive Smart Display was used to recreate
move on their fridge. This allows many playful types of
the feeling of tangibility
messaging between different parties1.
With another terminal, we had the touch sensitive aspect
The Prototype but we had to compromise on size and setting: a Smart
Based on the report from that project, Madelene Lindström Display with a touchscreen (Figure 7).
and I created an experience prototype to simulate what the
Despite the obstacles and limits posed by the lack of
experience may be like. The components included an
equipment, we were able to set up a representation of
application generated in Macromedia Flash containing
Poetry in Motion that recreated the experience of the
graphical ‘magnets’ which could be dragged around with a
product in insightful ways. This leads me to believe that the
mouse or input device. The application could be run on any
constraints involved in a Flash clip on a webpage does not
computer that was connected to the internet. A server was
have to take away from the experience value of a product.
set up with a Perl script that allowed the computers to
communicate with each other.
A Hypothetical Example
As an illustration, I will present a hypothetical
advertisement on a newspaper website for Poetry in Motion
that simulates the actual experience to a high degree.
As with the Sony Ericsson example before, at first glance,
the graphic at the bottom of the page appears to be an
ordinary banner ad that invites readers to click on it (Figure
8).
Figure 6. User evaluation of the prototype with the Powerwall.

1
Bogren, L. Internal Project Report: Poetry in Motion –
Sharing of everyday happenings, emotions and feelings in
an entertaining media

13
 What is recreated is in fact exactly what the product is in
reality: a shared space of words. Besides the obvious fact
that the interaction is not taking place on actual
refrigerators, the users get a very good representation of the
experience of Poetry in Motion.

Feasibility
The preceding example is by no means difficult to create. In
fact, the application that enables different parties to
communicate through the magnets already exists. It was
written for the physical prototype which I described in the
last section.

CONCLUSIONS
It should be clear now that it is quite achievable to recreate
the experience of a product with a representation of it to the
consumer in a similar way as we do to understand it
ourselves when we are designing it. The greatest obstacle
hitherto has been lack of affordable technology. Today, as
demonstrated by the Poetry in Motion project, there are
neither technological nor economical impediments to create
a virtual experience model.
It must be said that while in many cases it is an excellent
way to familiarize prospective customers with a product,
Figure 8. Interactive banner
the fact remains that some wares can only be experienced
through direct physical interaction. Even the example with
the Saab electronic key, while a nice preview, cannot
communicate the experience quite vividly enough. How
does opening the top on the 9-3 feel different from doing it
in a Mercedes-Benz? Clearly, there are limits to how much
can be reproduced to substitute actually using, feeling,
living a product.
That said, I believe that emerging technologies will make
this kind of marketing a viable complement to actually
Figure 9. The banner offers to invite friends from the user’s going to a shop and trying out merchandise. Most
buddy list to try out the Poetry in Motion together. importantly, it will shift the emphasis of marketing as
psychological manipulation to a more ethical practice of
Once the user does that, the banner area expands showing
straightforward evaluation.
an entire fridge and encouraging him or her to invite one or
more friends from an instant messaging buddy list to try it
ACKNOWLEDGEMENTS
out. It provides a standard invitation to send to one’s friends
I thank Niklas Andersson, Dipu Deshmukh and Bryant
(Figure 9).
Peltier for reviewing this paper, Linda Bogren for
consulting and helping us realize her project, Madlene
Lindström for building the prototypes with me and Oskar
Fjellström for helping with technical equipment.

REFERENCES
1. Buchenau, M. and Suri, J. (2000). Experience
prototyping. Proceedings of DIS '00, 424-433
2. Gao, J., Sheng, B., Chang, L., Shim, S. Online
Advertising - Taxonomy and Engineering Perspectives
Figure 10. Multiple users can send each other poetry and
phrases. 3. eBeam Interactive. http://www.e-beam.com

When they respond and connect, what we get is a sort of

chat room where everybody plays around with some given
magnets to send each other poetry and phrases (Figure 10).

14
 Capturing the Big Picture: a Design Case Study with an
Ethnographic Approach

Malin Mäki Mine Danışman Taşar

Umeå Institute of Design Umeå Institute of Design
Umeå University, SWEDEN Umeå University, SWEDEN
malin_maki@hotmail.com minedanisman@yahoo.com

ABSTRACT processing and to shorten the learning period that the

Understanding of the context of user environment and operators go through. While carrying out the research we
interaction is the key to product innovation and successful applied a holistic approach to examine both the
product design. In this study, we will discuss the benefits of organizational and the cognitive aspects. To do so,
applying an ethnographic approach to the design process ethnographic field study techniques were used, in particular
and also address the use of metaphors as a tool. We will participant observation and video recording. They were
focus on the benefits of involving ethnographic fieldwork helpful to understand the broader context of work practice
studies in the early stages of the design process, to capture and socio-culture as background for human actions [3].
the big picture before going on to a more detailed level. To Later on, our research led us to work with specific aspects
exemplify, we will use a design case study of information that appeared to be significant.
handling in specialized vehicles made in collaboration with
the forest vehicle industry. Our findings indicate that the METHODS
application of a holistic approach and various methods can Research methods used included literature studies,
help in the design process. participatory observations, software studies and low-fidelity
prototyping. Personas [4] were also utilized in order to get a
Keywords clearer picture of the people involved. An interview was
Design methods, ethnographic fieldwork studies, conducted in a harvester, in which the driver was video
metaphors, holistic approach, quick and dirty ethnography. recorded in order to be analyzed later.

INTRODUCTION The basis of this design case is to have a post-modern

Due to the importance that is given to high productivity, approach, by which we refer to the questioning of the
saving time and earning money are two essential areas of statement that there is an objective truth. By applying that
interest within the forest industry. Vehicle operators receive perspective you have to consider your own role, as a
a great deal of information in a limited time and evaluate researcher, and question your own goals and motives. It is
them as soon as possible to be able to transform time into critical to be reflective.
money. Therefore information handling in specialized For instance, in our case it was important to reflect over the
forestry vehicles gains considerable importance. Today impact our presence had on the way the operator worked
control systems are being used in harvesters which let and responded to us. By analyzing and reflecting on the
operators control the cross cutting process adapted to valid situation and have a critical standpoint, the researcher can
price lists for maximum yield. increase the reliability of the findings. Another good way is
Within this case study the main focus has been how to to go back to the users and ask them if they recognize
reduce the cognitive workload concerning information themselves and their situation within your analysis.
It is also vital to conduct the interview in the “right” way. It
has been kept in mind that the questions have to be
Permission to make digital or hard copies of all or part of this work for
personal ortoclassroom
Permission useorishard
make digital granted without
copies of allfee provided
or part of thisthat copies
work for are not
carefully constructed so as not to influence the interviewee.
made orordistributed
personal classroomforuseprofit or commercial
is granted without feeadvantage
providedandthatthat copies
copies are bear
not During the interview we formulated open ended questions
this notice and the full citation on the first page. To copy otherwise, or
made or distributed for profit or commercial advantage and that copies bear but still with a structure and within a frame. By doing so we
republish, to post on servers or to redistribute to lists, requires prior specific
this notice and
permission the full
and/or citation on the first page. To copy otherwise, or
a fee. were able to get the information “in-between the lines”.
republish,
UIDSC´05, to post on servers
January 13, 2005,or to redistribute
Umeå Instituteto of
lists, requires
Design, prior permission.
Sweden
Copyright 2005 Umeå University Collecting information from different resources and
.UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden
comparing different facts is another important research

15
Figure 1: During the field studies we had the opportunity to observe an operator in practice. We conducted an interview and a video
recording during the observation.

aspect. In our case, we not only gathered information from for throughout the research has been how to increase the
the field study and the interview with the operator but also validity and the reliability under such circumstances.
had lectures from SLU, Swedish University of Agricultural
Sciences. Moreover, our research included information Using Video
from forest industry, magazines and booklets. By Video filming was one of the methods that were utilized in
comparing and combining the gathered information, a good the research. The possibility of being able to go over it
overview was provided and the validity of the information repeatedly made it a useful tool. Although it can not be
was increased. claimed that we have explored all the possibilities of using
a video recorder, the gathered data through this method
Quick and Dirty Ethnography helped us to get a clearer picture of the environment and
Due to the fact that we have been trying to capture the conditions that the operator is working in. Jacob Buur and
user’s needs and apply a user-centered perspective to the his colleagues offer an interesting method, with “Video
design process, it has been quite helpful to use ethnographic Action Walls”, that can be useful to fully withdraw the
methods [2] in the research. This approach enabled us to information that is gathered with video [3]. In this
“see through the user’s eyes and be in their shoes”. To do technique 12 – 20 small-size video clips are shown on the
so we aimed to describe the reality through user’s screen. Each clip runs a loop of an action, therefore the
perspective and in that specific context. The interest has video wall not only shows dynamic information that still
been in qualitative data rather than a quantitative one. In images cannot convey, but also shows this information in
order to get this kind of information interviews and field parallel, which provides an opportunity to compare actions
observations were conducted, which are both prevailing that normal video can not.
methods in the field of Interaction Design.
Using Metaphors
"Typically, ethnography will take place over a period of It has been very useful to exploit metaphors as a tool during
several months with at least the same amount of time spent the whole design process. The metaphor has been a way of
in analysis and interpretations of the observations [1]." help focus and kept us on the right track. For example,
Many times it is not possible to spend months in the field while formulating and organizing the questions for the
collecting and analyzing data. Therefore, an approach interview we have utilized a funnel metaphor, which made
emerged to meet the increasing time demands which have us start with overall questions and narrow them down to
been called "quick and dirty" ethnography [5]. This more specific ones such as a funnel does. To do so the
approach includes shorter and more focused studies which purpose was to capture the contexual situation before going
can refer back to similar previous ones. into a deeper and detailed level.
We also chose to pursue our fieldwork study in a “quick Apartment metaphor was another metaphor that was
and dirty” way as the time was limited. In our case we had utilized. When building an apartment the first thing is to
the opportunity to make the field observation only once decide how the floor plan will look like before building the
which kept us from drawing too concrete conclusions from walls. The last thing is to choose the pattern on the
the gathered data (Fig. 1). We also could interview just one wallpapers. While designing the cutting control system, we
operator, who was not a typical harvester operator since he tried to see it as an apartment. Structring the system’s user
was working as a prototype tester. To be limited in your interface was prior to picking radius or shadows of a dialog
resources of information can sometimes be the case which box.
is not the ideal base for a design solution. What was strived

16
Using this tool throughout the design process simplified the The work is stressful, intensive and demands continuous
understanding of complex information. Familiar images concentration. There are several different kinds of control
from everyday life helped us to keep the track in solving systems in the cabin, like computer displays, traditional
problems. It has also been a support in our communication, control panels and two joy-sticks for controlling the crane
within the research team. Simple and well-known images arm and the cutting head. Each of the joy-sticks controls
can quickly create a common ground or an understanding of more then 35 functions. Today the period of training takes
a problem between the team members. It has worked as an at least six months and it can take up to one year until an
instant reminder to stay on the “right track”. operator becomes skilled.
To be a skillful operator, there are several areas that are
OBSERVATIONS AND FINDINGS
What we have seen through our research has been that it is important. First of all humans are different and so are the
important to look at the whole contextual situation not only way we process and learn new information. This means the
to the specific task of cutting trees (Fig. 2). In the forest amount and kind of information that a person can handle is
industry high productivity is very important so time and also different. There are some general rules, or guidelines,
money are two keywords. In almost every case there are at that are applicable on humans and the cognitive processes,
least two to three drivers that are driving the same machine, which could be helpful when focusing on the cognitive load
since it is important to run it as many hours as possible to concerning information handling in a professional context.
make profit. Because they are working in teams they are
CONCLUSIONS
dependent on each other. They need to have a good
Drawing on ethnographic fieldwork studies has been a
communication within the team. The planning phase is
powerful tool. Everything within the context was dependent
especially important in order to work in an efficient and
on each other so in order to reach the solutions for details,
profitable way.
the big picture had to be comprehended first. For instance,
through examining the facts from various aspects we found
out that improving only the cutting process would not be
enough to increase the effectiveness of the operator.
The holistic approach showed us that the task had to be
considered as a whole story from the cabin that the cutting
decision is made to the mill where the logs end up. The
research revealed that the team’s cooperation was a strong
point within this context. This, for example, directed us to
load extra functionalities to the memory card, the device
some of the operators have been using to store the cutting
data, so as to strengthen the communication between the
team members. Different methods that we used supported
each other and provided us with information that we could
rely on.

DISCUSSION
As mentioned earlier it is crucial to take in the contextual
situation in order to be able to fully understand the user’s
needs. However, it would also be interesting to stretch the
approach further into the design process since the analyzing
phase of gathered data is as important as the actual field
research. In further work it would be worth trying to
explore the potential of using video not only as
documentation tool but as a tool in the creative phase as
Figure 2: Storyboards and personas were used to
understand the context of the operators working situation.
well.
Utilizing metaphors proved to be very helpful in the design
process, not only to understand complex information but
While driving a harvester the operator has to combine also as a way of communicating with each other. This is
driving and maneuvering with other tasks that come with something that we will definitely keep on using in the
forestry work. The operator has to take in tremendous future and try to find ways to explore it even more.
amounts of information while handling the machine. These
additional tasks demand a high cognitive workload and Understanding of the context of user environment and
extensive training is often required to be able to handle the interaction is the key to product innovation and successful
different modes of interaction. product design. Undoubtedly, the quick and dirty

17
ethnographic approach will keep on growing in the field of 2. Blomberg, J., Giacomi, J., Mosher, A., & Swenton-
Human Computer Interaction, HCI. This approach can be a Wall, P. Ethnographic Field Methods and Their
way of meeting the demands on the market for faster and Relation to Design.In D. Dchuler and A. Namioka
cheaper product development. (Eds.) Participatory Design: Principles and Practices.
Erlbaum: New Jersey, 1993.
ACKNOWLEDGMENTS 3. Buur, J., Jensen, M.V., Djajadiningrat, T. Hands-Only
We would like to thank all the people and organizations that Scenarios and Video Action Walls – novel methods for
contributed to this work; Komatsu Forest, Bengt-Arne tangible user interaction design, Proc. DIS ’04. (2004),
Walldén, SLU, Iwan Wästerlund, Dianne Wästerlund, 185-192.
Carin Larsson, Niklas Andersson, Mike Stott, Dennis
Holmgren, Mikael Sundling and Fredrik Sundling. 4. Cooper, A. The Inmates Are Running the Asylum:
Why High Tech Products Drive Us Crazy and How to
REFERENCES Restore the Sanity, Sams Publishing, Indiana (1999).
1. Bentley, R., Hughes, J. A., Randall, D., Rodden, T., 5. Hughes, J., King, V., Rodden, T. & Anderson, H. The
Sawyer, P., Shapiro, D., & Sommerville, I. Role of Ethnography in Interactive Systems Design.
Ethnographically–informed systems design for air Interactions, April, 1995. 57-65.
traffic control. Proceedings of the Conference on
Computer Supported Cooperative Work. (1992), 123-
129.

18
 “Experience Portfolio” - Using Interaction Tools
to Explain the Concept of Interaction Design
Paula Guntaur Kristofer Öberg
Umeå Institute of Design Umeå Institute of Design
Östra Strandgatan 30 Östra Strandgatan 30
Umeå, SE-901 87 Sweden Umeå, SE-901 87 Sweden
+46 (0)90 77 89 22 +46 (0)90 77 89 22
paagur03@student.umu.se krrobg03@student.umu.se

ABSTRACT expressive rendering or sophisticated virtual 3D model

In this paper, we describe the “Experience Portfolio” as a cannot communicate a designed experience. This dilemma
valuable tool that utilizes interaction design tools to explain has led us to brainstorm on a method that would allow a
the concept of interaction design. Today’s public does not person to visualize the work of an interaction designer. We
have an understanding for the role of the emerging constructed an “Experience Portfolio” for the interaction
interaction designer. We created a designed experience design students in our class. The term portfolio refers to a
that would promote and define an interaction design collection of ideas and projects assembled by the individual
project. In order to achieve our concept, we tested two designer to exhibit his/her skills. Users will benefit by
low-fidelity prototypes using the Wizard-of-Oz method. interacting with a designed experience while visualizing
We can conclude that the experience in combination with interaction projects on a dynamic medium.
the seamless integration of technology highly enhances the CONTEXT
user’s comprehension of the interaction design field. We intend to show the project at the UID (Umeå Institute
Keywords of Design) final degree exhibition. We feel that this project
Experience prototype, low/high-fidelity prototype, will introduce the notion of interaction design to friends
phidgets, interaction design tools, tangible user interface, and family of the exhibitors before they enter the display
user-testing area.
INTRODUCTION There are two factors which influence how well the
Over the past twenty years, we have seen processor speeds “Experience Portfolio” communicates the concept of
double, hard drive capacities increase to unusable amounts interaction design to our target audience. The first being
of storage, unlimited versions of new operating systems the content of the “Experience Portfolio” which will
and shrinking hardware components. This phenomenon include the student’s final thesis work. The second factor
has allowed computer technology to leave its physical being the interface or the manner in which the content of
container, the grey computer box, and appear in various the portfolio is presented. Due to time restraints of three
products. The creation of these new hybrid objects led the weeks, we chose to focus on the interface which in itself
interaction designer to emerge borrowing traditional form illustrates notions of interactivity and tangibility.
and conceptualization techniques from the industrial design EXPERIENCE PROTOTYPE
profession, programming skills and computer knowledge Buchenau and Fulton-Suri describe the experience
from the computer scientist, and communication and prototype as a form of model making that enables designers
information expertise from the graphic designer. The and users to gain appreciation of existing or future
convergence of these domains will breed a designer who conditions through an active engagement with a prototype
integrates technology seamlessly into products focusing on [1]. Users employ their visual, tactile, and auditory senses
the product as an experience rather the product as an to identify with the experience. This sensorial connection
object. along with the temporal and contextual qualities of the
Experiences are intangible. A beautiful prototype, prototype can greatly enhance understanding from the user.
Ishii and Ullmer argue that tangible objects lead to a much
Permission to make digital or hard copies of all or part of this work for
personal or classroom richer multi-sensory experience of digital information [3].
LEAVE BLANK use THE is granted
LAST without
2.5 cmfee(1”) provided
OF that
THE copies
LEFTare not
made or distributed for profit or commercial advantage and that copies bear Their work with Tangible Geospace incorporates tangible
COLUMN
this notice ON THE
and the full citation FIRST
on the PAGE
first page. To copyFOR THE or
otherwise, user interfaces as a means to visually and haptically interact
COPYRIGHT
republish, to post on servers NOTICE.
or to redistribute to lists, requires prior permission.
with geographic spaces. Similarly, our “experience
portfolio” will entice the user to envisage interaction design
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden

19
projects by navigating with physical objects and seeing the Conclusions – Low-Fidelity Prototype 1
results projected on a large surface. We tested three students and one staff member from the
UID. The low fidelity prototype informed us of several
RESEARCH GOALS
flaws. For example, the fence we used to mark the area
We consider the “Experience Portfolio” to be a reflection
inhibited the user from approaching the platform.
on promoting and creating interest for the interaction
Secondly, the context where we mounted the installation
design field. We describe it as a design experiment rather
was cluttered and disordered. It was unclear that the magic
than a traditional research project.
wand belonged to the “experience portfolio”, more less a
Primarily, our goal is to evaluate the effectiveness of an tool to navigate with. Also, the magic wand forced the
“Experience Portfolio” in aiding users to understand the users very close to the life-size projections inhibiting them
interaction design field. Often, portfolios are viewed by to get an overall view of the animations. The most
means of physically flipping through pages in a book important observation was that users felt awkward poking
and/or browsing a website. However, to enhance the images of humans with a magic wand.
experience in our “Experience Portfolio”, we use tangible
LOW-FIDELITY PROTOTYPE 2
user interfaces which incite interest, exploration and
manipulation. Inspired by Ullmer and Ishii’s work on phicons (physical
icons), we associated each student animation with an
Secondly, we will evaluate if the Wizard-of-Oz low-fidelity arbitrary icon [4]. The projected surface shows eight
prototyping method is useful in the development stages of people each holding a different object which corresponds
an experience prototype. Molin describes how this to a physical icon. This direct visual clue creates a clearer
technique can improve the interactive design system for it connection between the tangible user interface and the
allows cooperative design and testing of computer interaction virtual projection. For instance, if the user wants to
early in the process [5]. This procedure lets users interact interact with the girl holding the “markers”, s/he should
with an interface without knowing that the responses are pick up the physical cube with the “markers” icon.
being generated by a human, not a computer. Moreover, this situation distances the users from the
LOW-FIDELITY PROTOTYPE 1 projected surface giving them a better overall view whilst
Svanæs and Verplank suggest magic and paranormal indicating where they should stand.
phenomena could be appropriate as metaphors for tangible Environment
user interfaces [6]. They argue that any sufficiently We situated ourselves in an isolated room with a projector,
advanced technology can feel like a magical experience. a computer, a table, eight phicons and one box. We chose
We used a “magic wand” as a metaphor for the tangible a smaller environment with less visual clutter. The phicons
user interface of our first prototype. were placed on a table two to three meters from the
Environment projection area.
We situated ourselves in a corner of a large studio. We Input Device
used a back-projection wall, a projector, a computer, a We simulated the behavior of RFID (Radio Frequency
magic wand, a paper platform and a fence to mark the area. Identification Tag) chips and readers. We created eight
Input Device phicons by crafting eight identical cubes and fastening
The technology we intended to simulate was the eBeam. different images to them. When the user picked up a
This system combines ultrasound, infrared, and wireless phicon, it activated a projected animation. We fabricated a
technologies with software to create a projected interactive large box with an opening the exact same size as the one of
display. The stylus pen lets the user interact directly on the the phicons. When the user inserted the phicon into this
projected image. box, it initiated the student’s portfolio.
Output Device Output Device
We created a quick FLASH demo. FLASH is a vector We used the same FLASH demo as the previous low
based animation program which allows designers to fidelity prototype. We added eight icons made of paper
quickly build interfaces. We photographed eight random which we taped to the projected surface. Once again, we
people, taking three to four images of each person. We relied on the Wizard-of-Oz method to make the user
imported the images into FLASH sequencing the still believe that it was their physical manipulation of the
portraits to achieve an animated movie clip. When the user phicon triggering the student’s animation and portfolio
tapped a person’s projected image with the magic wand, we display.
manually pressed a key on the computer to activate the Conclusions – Low-Fidelity Prototype 2
person’s animation. This movie clip lead to the student’s We tested four students from the UID. Most users
interaction design portfolio. understood how to reach the portfolio page. However, the
connection between the screen and object was very unclear.
The prototype was perhaps “too” low-fidelity. The icons

20
were glued onto pink foam, which was an unpleasant
material to manipulate.
Moreover, it was slightly more difficult this time to
simulate the technology with the Wizard-of-Oz method.
The users manipulated the phicons quicker than we were
able to press the appropriate keys on the computer.
Based on the results from the two Wizard-of-Oz low-
fidelity prototypes, we felt that only a fully functioning
high-fidelity prototype would allow us to evaluate if the
“Experience Portfolio” communicated the concept of
interaction design to our users. Figure 2. Master of Interaction Design Class
HIGH-FIDELITY PROTOTYPE Environment
We built the high fidelity prototype. We chose symbols for The life-size projections were viewed on a large, clean
the phicons that were meaningful and relevant to the white surface in the UID’s exhibition hall. The hidden
interaction design students. We explored our workspace projector was connected to a laptop running the
and photographed eight commonly used interaction tools. projections.
1) Laptop computer to make FLASH and Director Input Device
simulations We used RFID tags and readers from the phidgets library
2) WACOM board as an additional sketch tool and developed by Greenberg and Fitchett [3]. Phidgets are
input device for the computer easily programmable sensors combined with customizable
3) Digital camera and recorder to video process software. We masked one RFID chip in each of the newly
4) Markers to give expression to storyboards constructed wooden phicons. We embedded eight RFID
5) Sketchbook to keep ideas readers in a custom made base. We added one RFID
6) Mouse as an input and navigation device reader to a separate custom made base which acts as the
7) Headphones and microphone to experience sound “portfolio drop box”.
8) Roll of tracing paper to sketch possible scenarios

Figure 3. Phicons
Figure 1. Interaction Design Tools
Output Device
We photographed the students in our class each holding We created 8 flash movie clips and programmed them to
one of these items. We shot several pictures (15-20) of activate on key presses. We used this information to write
each student performing an action. We cropped and a program in VISUAL BASIC that associated each RFID
sequenced these images to create an animation. In chip reading with a key press. Therefore, the movement of
addition, we created a portfolio page for each student. the RFID chips in relation to the RFID readers
automatically generated different computer key presses.
There are three possible configurations:

21
 Most users thought the animations were amusing, thus
provoking interest to explore and discover the rest of the
“experience portfolio.” Users were confident in stating that
this project presented them with a clear boundary between
a product as an experience (interaction design) and a
product as an object (industrial design).
GENERAL CONCLUSIONS
The final “Experience Portfolio” proved to be effective as
it successfully provided users with a relevant subjective
interaction experience. The user’s active participation in
the portfolio gave them a new perspective on how
technology can pleasantly enhance everyday objects. The
connection between the phicon and the projection was not
always evident at first. However, this uncertainty quickly
1) When the user picks up a phicon, the full-scale projected disappeared after one or two phicon manipulations.
student engages in motion. The Wizard-of-Oz prototyping technique was quick and
efficient to understand technological behavior in new
contexts. We were surprised to discover that our first low-
fidelity prototype was unsuccessful with the users. In
theory and on paper, the concept was logical and worked
well. However, in practice we uncovered many
irresolvable issues. The Wizard-of-Oz method proved
effective to evaluate the intuitiveness of the portfolio’s
interface. It is a useful tool in the development stages of an
experience prototype.
FUTURE WORK
We intend to bring the “Experience Portfolio” outside of
the university and test the prototype with users of various
ages and from different professional backgrounds. We
believe this would give us more accurate results.
2) When the user inserts the phicon in the “portfolio drop
ACKNOWLEDGEMENTS
box”, the student’s portfolio appears in the projected area.
We would like to thank Oskar Fjellström, research engineer
at UID, for his patience and tremendous help programming
the phidgets.
REFERENCES
1. Buchenau, M., Fulton Suri, J. “Experience
Prototyping”, In Proceedings of the conference on
Designing interactive systems, p.424-433, ACM 2000.
2. Greenberg, S. and Fitchett, C. (2001) Phidgets: Easy
Development of Physical Interfaces through Physical
Widgets. In Proceedings of the UIST 2001, p209-218,
ACM 2001.
3. Ishii, H., Ullmer, B. “Tangible Bits: Towards Seamless
Interfaces Between People, Bits and Atoms”, In
Proceedings of the CHI ’97, p.234-241, ACM 1997.
3) When the user returns the phicon to its original place, 4. Ishii, H., Ullmer, B. “The MetaDESK: Models and
the animation ceases. Prototypes for Tangible User Interfaces”, In
Proceedings of the CHI ’97, p. 223 - 232, ACM 1997.
Conclusions – High-Fidelity Prototype
We left the installation running in the exhibition hall for a 5. Molin, L. “Wizard-of-Oz Prototyping for Co-operative
few hours and observed several users. It took few seconds Interaction Design of Graphical User Interfaces”, In
for each user to understand that manipulating a phicon Proceedings of the NORDICCHI ’04, ACM 2004.
triggered a reaction in the projection area. The users 6. Svanæs, D., Verplank, W. "In Search of Metaphors for
immediately deciphered how to reach the portfolio page. Tangible User Interfaces", DARE'2000, ACM, 2000.

22
 Applying Cognitive Science Research in
Graphical User Interface (GUI)
Gabriel Åberg Jessica Chang
Umeå Institute of Design Umeå Institute of Design
Storgatan 82 Pedagoggränd 11A 102
S-903 33 Umeå S-907 30 Umeå
gabriel.aberg@home.se jesschang00@yahoo.com

ABSTRACT various conceptions of design principles and guidelines,

In this paper, we will discuss the usage of cognitive science there still lacks a clear direction for graphical user interface
research in graphical user interface. In the area of interface design in operator interfaces. One obvious reason could be
design, there is a lack of guidelines based on actual that different industries require specialized functions from
cognitive science research. We intend to explore this area one another; thereby difficult to apply any specific set of
by focusing on the forestry industry and forestry operators, interface design guidelines.
in particular, and how their visual cognitive work load
affects their learning process, attention span and decision The operators are typically concerned about whether the
making. We will examine operation interfaces, where final machine is running smoothly and accurately, and the
production is dependent on an operator’s work efficiency production count. While this may sound uncomplicated,
and collaboration with the computer interface. Using the reality is far from. With production dependence on the
information found in cognitive science research, we will operator and machine cooperation, the operator is inundated
design a graphical user interface as an alternative to the by the amount of cognitive work. Cognitive work is work
existing tree harvester computer interface. By aiming to primarily having to do with problem solving and decision
lessen the operator’s cognitive task load, this paper explores making. To lessen an operator’s cognitive task load, one
the possibility of incorporating cognitive science research area to consider is the research of cognitive science.
into the human computer interface (HCI) design.
FORESTRY OPERATOR’S TASK ANALYSIS
Author Keywords
Our study focuses on the sequence where the operator fells
Graphical user interface (GUI), human computer interface the tree, de-limbs it and cuts it into logs. Although the
(HCI), cognitive workload, visual cognitive effort, learning, sequence in itself takes less than 30 seconds, it is repeated
attention, memory relief, decision making. continually. Here is also where the majority of cognitive
tasks are in-demand. During these thirty seconds the
INTRODUCTION operator has to make several crucial decisions. For instance
Many designers and researchers have examined the these decisions involve estimation of species and quality
complex world of interface design. With technology selection as well as deciding whether the system is making
increasing in the workplace, various industries rely on correct calculations.
operators working in machineries that involve a computer With overwhelming amounts of information presented to
graphical user interface (GUI). For example, the the operator simultaneously, this paper focuses primarily on
automotive, electronic equipment, forestry, mining, metal the operator’s visual cognition. The visual sensory
machining and fabrication, printing & publishing, and occupies 75% of human sensory input, and is undoubtedly
textiles, are some industries where workload is placed on a of great importance for forestry operators, and operators in
machine operator to meet production needs. Even with general.

Sequence Run Through

Ninety-seven percent of forestry operators in Sweden are
male therefore our study will be based on as such. As an
Permission to make digital or hard copies of all or part of this work for operator enters the machine he sits in front of the GUI, logs
personal or classroom use is granted without fee provided that copies are not in to the system and is led to his personalized settings.
made or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. To copy otherwise, or
These settings, among others, consist of chair positioning,
republish, to post on servers or to redistribute to lists, requires prior permission. joystick sensitivity and movement, and speed of the boom –
settings make the operator’s work easier by making the
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden system personalized for individual behavior.

23
Visual Cognitive Task Analysis Learning Research
The operator is repeatedly faced with visual sub-tasks Cognitive psychology research shows that search time was
which interrupts the work flow. Both, visually, in focus as shorter with symbolic coding than with alphanumerical
well as distractions, interfere with his work. When name coding when less than 49 objects are involved [9].
examining the complexities between the operator’s tasks (Figure 1)
and collaboration with the information handling system, we
A second research of relevance to learning is the idea of
categorized the visual cognitive task analysis into three
abstract theory, which states that human reason using a set
sections. Learning, attention and decision making; three
of abstract, logic like rules that can be applied to any
subjects within cognition we found imperative for operator
domain of knowledge [4]. Our interpretation of this theory
task analysis. We point out hindrances within each
led to the idea of sequencing. Applying the idea that one
category as areas in which to improve in our final interface
event leads to the occurrence of a second event.
design, and quick solutions for each. [7]

Learning
The complexity of the operating system causes memory
overload on the operators. It is an intricate program
containing enormous amounts of information. For
instance, the 2 joysticks alone, each houses over 30
functions. An experience operator pointed out that it is
common for inexperienced operators to use the trial and
error technique; which is an inefficient [9] and potentially
dangerous way to operate heavy machinery.
One way of providing cognitive relief in learning would be
to adopt the idea of “what you see is what you get”. If
relevant information is present without memory recall,
perhaps some system complexity can lessen.

Attention Figure 1: Subjects were quicker at identifying correctly

While the operator carries out his work, frequent through graphic symbols than with words.
disturbances emerge. He becomes unfocused by activity
switching, distractions and overload of visual information
[6].
Solutions could involve giving the operator access to instant
recall. For instance, reminders or task-logs, where all
activities performed are recorded and easily accessed.
Usage of color could be another area to consider.

Decision Making
There are simply too many decisions an operator has to
make [10]. Not only does he make decisions about what he
sees, i.e. the chosen tree and its characteristics, he also
makes decisions of information to input into the system; all
this is done simultaneously. Faced with so many decisions,
ultimately it is up to the operators which ones to choose.
Figure 2: Pictograms versus alphanumeric symbols
If choices and hints are easily available, it may help
quicken the decision making process.

RELATED COGNITIVE SCIENCE RESEARCH Attention Research

The three defined categories led to the aim to reduce visual Most readily available research within the three categories
cognitive effort and provide memory relief. Looking into we found was the study on attention. Initially, reasoning
cognitive science research that shows relevance to the three that the most important information is usually bigger and
categories of learning, attention and decision making; the brighter, and less important information is smaller and less
following are some of our findings which we deem helpful bright. Now we have learned much more about how
in directing our GUI design. attention can be manipulated.

24
 color to use to grab our attention in the periphery. For
instance, in recent years, police car sirens have changed
from red to blue.

Decision Making Research

In accordance with the Explorer interfaces, hints can help to
increase performance [6]. This is not a scientific fact, but
an example that most Explorer users would agree on. A list
of previously visited websites matching the beginning
letters of the currently wanted website appears when surfing
the world-wide-web. This eases the decision making
process as well as memory recall.

FINAL DESIGN – GUI

After extensive analysis of all research material, we
sketched numerous preliminary design proposals. The
intention is to use relevant research and apply them in our
GUI. Following are some illustrations of our design
sketches.

Proposal of GUI to Help Learning

Figure 3: Visual fields
To use the researched information which implies that
images or symbols can be easier to distinguish than words
Images Vs. Words
In a 40 millisecond words versus complex geometric or examples for using graphic symbols instead of
faces test, words are more easily reported when in the right alphanumeric symbols were done.
visual field and images in the left visual field [1]. (Figure Figure 4 uses color, graphic symbols, as well as the idea of
3) This is a cognitive psychology research which states sequence of events to help operators in his tasks. This
that words are identified quicker when they are in the example adopts the idea that if the current action is largest
subject’s right visual field; and images are identified in presentation, then the operator will not be distracted by
quicker when presented in the left visual field. irrelevant information. After each action passes, the
correlating information moves over to the right and
Left-To- Right Theory becomes smaller in presentation.
Well recognized in the western world is the awareness that
people tend to approach written information from the left to
the right. When reading a magazine or off a monitor, for
instance, a majority of the population would direct his/her
gaze to the top left area initially then sweep to the right.
Eventually, he/she finishes reading in the lower right-hand
corner. This tendency is used frequently by the newspaper
and advertisement industry when arranging lay-outs.
Location on the page plays a big part in determining the
price.

Color Influence
The above theory can be overridden if a larger, brighter, or
distracting image is present somewhere on the page. That
image would then become the main focus on the lay-out. In
most cultures the color red communicates warning and stop,
whereas green is used to communicate safety and go. What Figure 4: Event sequence interface
about other colors we wondered? In fact, studies show that
color has no effect on operator performance, but serve as
personal preferences.
Another interesting cognitive science research explains the
theory of “blue peripheral vision” [11]. This theory asserts
that because the cones in our retina which identifies the
color blue are on the outer edges of the eye, blue is a good

25
Proposal of GUI to Help Attention
The sketch in figure 5 utilizes the study that graphic
symbols are more easily recognized when placed in the left
visual field and alphanumeric symbols in the right visual
field. Figure 6 demonstrates the “logging” reminders,
where previous activities are recorded in a chronological
order; whereas figure 7 illustrates the logging as well as the
use of the blue color to highlight peripheral information.
Finally, figure 8, are various examples of how color and
borders can affect attention.

Figure 8: Visual effect from colors and boarders

Figure 5: Symbols vs. text

Figure 9: Multi layering

Figure 10: Most important info. on top

Figure 6: Logging

Figure 11: Easy access to additional info.

Sketches of GUI to Decision Making

The following sketches (Figure 10) show a GUI with the
idea of multi-layering. The most important information is
Figure 7: Peripheral information highlighted in blue
on the top-most layer, visible at all times, whereas the less
important or relevant information is on the second and third

26
layer. For instance, the pricelist information and machine DISCUSSION AND FUTURE WORK
maintenance information can be accessed easily when More work is needed for us to reach an explicit conclusion.
needed. Due to the shortage of time, we were unable to carry out
user testing. We want to point out our awareness that user
testing is crucial in any GUI development and design, in
order to determine the validity of the design. Without
proper user studies and tests, we cannot reach conclusive
results.
When the time comes for user tests, some questions of
relevance to keep in mind are for instance “is this user
reliable?”, “will this test give relevant feedback to the
project?” and “how should this test be performed so that we
don’t influence the user and thereby the test result?”

In addition, the importance and value of operator expertise,

opinions and comments can not be emphasized enough.
The ideal situation would be to eventually perform user
tests with actual users. Something to keep in mind is that
Figure 12: Final GUI the level of expertise in users may lead to changes in the
design criteria. In our case, this can potentially make some
RESULT of our design concepts invalid.
We combined various parts of our research to create the
final GUI. First, we based our design on a previously ACKNOWLEDGEMENTS
looked-at research which showed color had no effect on We thank SLU, Komatsu Forest, our tutors, and all
operator performance. For this reason we decide not to let cognitive science publications that wrote and provided
color deter from our intentions, therefore a monochromatic helpful studies for our project.
neutral color is chosen. Please see figure 12 below to see
our final GUI. REFERENCES
1. Kolb, Bryan & Whishaw, Ian Q. Brain and behavior.
CONCLUSION Worth Publishers, New York USA, 2002.
When giving shape to our GUI we used our findings of
2. Corker, K.M. “Human Performance Simulation In
placement, color, size etc. to create a rough outline of
The analysis of Advanced Air Traffic Management”.
content designated areas within the GUI. We then
Winter Simulation Conference ’99, 821-828.
organized the rest of our research ideas by first creating
separate design concepts, then combining them in the end. 3. Davis, G.I. and Buck, J.R. “Simulation-Aided Design
Structuring our initial design work in this way, gave us the of Man/Machine Interfaces in Automated industries”.
opportunity of trying out different combinations of various Winter Simulation Conference Proceedings ’81, 407-
findings. Keeping in mind that not all researched theories 416.
and materials can be applied simultaneously; always
striving for the most promising solution while making 4. Eysenck, M.W. and Keane, M.T. Cognitive
compromises amongst gathered materials. Psychology. Psychology Press 2003, East Sussex, UK.
25-80, 119-149, 185-211, 440-443, 445-456.
To know which set of information from research to use and
where to use them are the biggest dilemmas in this type of 5. Foley, J.D., van Dam, A., Feiner, S.K., and Hughes,
project. Even while experimenting on ourselves, we J.F. Computer Graphics: Principles and Practice, 2nd
discovered some contradictions in the GUI. We are led to Ed. Addison-Wesley Publishing Company, 1991. 601-
believe that much more project specific work is needed in 603, 618-621.
the field of interaction design. To help designers make 6. Lampson, B.W. and Sproull, R.F. “An Open Operating
better use of cognitive science research in particular, more System for a Single-User Machine”. Proceedings of
guidelines are needed. Although each case study looked at the seventh ACM symposium on Operating systems
within this project is based on scientific methods, we need principles ’79, 98-105.
to keep in mind the results are project specific. As in any
type of research, it is essential to always bear-in-mind to 7. Nemeth, C.P. Human Factors Methods for Design:
take the information presented with a grain of salt. Making Systems Human-Centered. CRC Press 2004,
London, UK. 195-217.

27
8. Quek, F.K.H. and Petro M.C. “Human-Machine
Perceptual Coorperation. Interchi ’93, 123-130.
9. Hyönä, J., Radach, R., Deubel, H. The Mind’s Eye:
Cognitive and Applied Aspects of Eye Movement
Research. Oxford: Elsevier Science, Amsterdam,
Netherlands, 2003.
10. Somervell, J., McCrickard, D. S., North, C., and
Shukla, M. “An Evaluation of Information
Visualization in Attention-Limited Environments”.
Eruographics – IEEE TCVG Symposium on
Visualization 2002, 211-215.
11. The Rods and Cones of the Human Eye.
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/
rodcone.html#c3b

28
 Designing a Tangible User Interface for
a Refrigerator for a Future Market
Hyoung Won
Department of Interaction design
Umeå University
SE-901 87 Umeå, Sweden
hyoung@hyoungwon.com
+46 703 593718

ABSTRACT
In this paper, mainly the design process used for designing
a Tangible User Interface (TUI) of a refrigerator for a
future market is explained, and partly how a TUI can
influence our daily lives is illustrated. Web surfing was
used as a sub research method to support ethnographical
research. Based on feasible scenarios and user research, a
TUI and a Graphical User Interface (GUI) were designed to
be tested by subjects in user test stage. Also, low-fidelity
prototypes and interactive animations were made for user Figure 1: Control knobs based on a TUI
tests. The TUI made the interactions between a refrigerator
and the user abundant, and many subjects liked this
interface, which saved them time and effort.
Author Keywords
UI design, refrigerator, TUI, design process
INTRODUCTION Figure 2: Control knobs based on a GUI
Unlike information appliances, household appliances
mainly had been controlled by only a TUI (Figure 1), but
nowadays a GUI (Figure 2) starts taking over the role of
manipulating the appliances from a TUI [1]. As household
appliances tend to connect and communicate each other, the
demand for a TUI is increasing. The design approach for a
TUI of household appliances for a future market must differ
from that for information appliances for a current market.
Therefore the ethnographical research, which is wildly used
nowadays as a research method in designing, but has some
defects - the lack of objective views, anticipation for future,
and propriety in designing technology-centered products -,
needs to be reviewed by designers. Figure 3: Design process
In this paper, through the design process (Figure 3) of PRE-STUDY
futuristic refrigerator’s TUI, modified research method and The main aim of research stage was to keep an objective
design approach are suggested. view. Thus, beside of ethnographical research, amateur
photographs of refrigerators in homes were collected via
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are not web-sites containing picture collections uploaded by
made or distributed for profit or commercial advantage and that copies bear ordinary people living in South Korea [2] and the U.S. [3].
this notice and the full citation on the first page. To copy otherwise, or Many of the pictures showed similarities, information
republish, to post on servers or to redistribute to lists, requires prior permission. reminders and memorable pictures on a refrigerator, in
using a refrigerator between the users in South Korea and
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden

29
 Figure 4: Pictures from South Korea (left)
and the U.S. (right)
those in the U.S. despite their cultural differences (Figure Figure5: The refrigerator in a Swedish family,
4). and a tangible magnetic holder
RESEARCH • Problem two - the children would like to put their
One Swedish family was contacted for ethnographical drawings on the refrigerator door, but they could not due
research. The family sent the picture (Figure 5) of their to a limited space.
refrigerator and exchanged e-mail with the author of this
paper. They gave the author some information on what are • Problem three - the contents of some information such
on their refrigerator and how they manage them. as funny story, and articles from the newspaper belong to
daily information. However, they had been on the
What they put on their refrigerator was: refrigerator for several months.
• Schedules for children, such as school schedule • Problem four - the parents and children needed to see
• Funny stories and comic strips from the newspaper the schedule of children’s school together, but the
• Newspaper articles schedule was put in too higher position, so children could
not see them.
• Sketches and drawings done by children
Problem from ethnographical research in designing
• Memorable pictures of family members futuristic products:
How they manage their information on the refrigerator: • Problem five - the futuristic refrigerator must have
• More important information was positioned higher and more interactive User Interface (UI) than that today’s
on top of other information. Also, it had been announced refrigerator has.
along with a vivid color alphabet letters. Based on research stage, several problem scenarios were
• Usually, short-term information was written on the made in a cartoon style to show problems more
post-it and long-term information was printed out and comprehensibly.
taped on to the refrigerator door. CONCEPTUALIZATION - suggesting solutions
Refrigerator – an information appliance After problem definition stage, several solutions were
According to the pre-study and research, most of proposed through brain storming, mind mapping, and web
refrigerators in a domestic area were used as an information surfing.
board. For solving problem two, four, and five, the refrigerator
Different users with different needs which has a big touch screen door in the right and a normal
In the Swedish family, their refrigerator was used by two refrigerator door in the left was ideated. The touch screen
different age groups - parents and children. The parents door also has a built-in scanner. Thus, the users can sketch
were always involved in informing, displaying, and drawings directly on the touch screen, and scan children’s
discarding information. The children put their water color drawings and printed information. This solution allows
drawings, and even sketched drawings on the surface of users to have the interaction occurred between the normal,
their refrigerator door directly. tangible door in the left and a screen, virtual door in the
right
PROBLEM DEFINITION
Problems found in research stage were: To solve the problem one, three, four, and five, the idea of
a personalized GUI was employed. This method can save
• Problem one - the users could not find out information users time to reach their information without unnecessary
immediately, because there was too much information on searching, and space. For instance, when the father
the refrigerator.

30
 Figure 6: The low-fidelity prototype for first user test Figure 7: First user Figure 8: First user test
approaches the refrigerator closely, the refrigerator test environment
recognizes him and shows him pre-decided information,
which only the father can access. For the children, their
information can be shown in the lower position of the door
screen according to their heights. Moreover, for the elderly,
the GUI on the screen is more vivid, and big alphabet
letters are used, due to their weak eyesight. Over time, the
color and size of information become faded out and smaller
in order to give the user the sign, this information is getting
out of date. Every day, the users can see the new comic
strips, funny stories and newspaper articles on the screen.
The solution to problem five was a tangible magnetic
holder (Figure 5), a tangible token [4]. This concept was
Figure 9: The second prototype Figure 10: Second user test
from the magnetic memo holder on the refrigerator, shown
in the pictures in pre-research stage of this paper. The users After drawing, he drags his sketch to an e-mail tangible
employ this tangible magnetic holder as a magnetic memo magnetic holder to attach this sketch to his e-mail in order
holder on the left door. On the right door, it is used as a to send his picture to his friends. The interactions between
program launcher. Also, it contains a small size program an email tangible magnetic holder and a sketch tangible
such as a sketch program for children, an emailing magnetic holder are occurring.
program, and so on. Thus, with these tangible magnetic FIRST PROTOTYPE
holders, the user can launch a program immediately without Low-fidelity prototypes (Figure 6) using paper models and
following complex, hierarchical menu structure. Macromedia Flash animations were made for user tests, and
SCENARIOS subjects were told about the definition of a tangible
Two scenarios for two family members in the story board magnetic holder. The valuations of these user tests relied on
format were created to show how the family members can subjects’ comments.
use this refrigerator in their daily lives, and solve the Four subjects participated in first user test. The test
problems shown in problem definition stage. environment (Figure 7) in first user test was in a small room
Scenario one with a big white board. A beam projector was employed in
Sara is a high school student. She orders a pizza by using a order to show real size GUI menus and icons to subjects.
tangible magnetic holder. Firstly, she places the tangible The scenario one and two were run in two different
magnetic holder, given by the pizza store near by her situations, one was following many steps of menus in a
house, on the refrigerator. The web site of the pizza store hierarchical menu structure and the other one was
appears on the door screen directly. When she orders a employing a tangible magnetic holder which leads the user
pizza, she scans a paper discount coupon to get a discount. to reach the specific menu directly (Figure 8).
Scenario two SECOND PROTOTYPE
Johan is an elementary student. He uses the tangible After the first user test, according to the users’ feedback, UI
magnetic holder, given by KFC as a free gift, for sketching. structures, menus, and icons of the first prototype were
modified and redesigned as the second prototype (Figure
9).

31
 LED

Cover

A screen door
A normal door Name tag 45mm

Wireless
device

Magnet 20mm

Figure 11: The refrigerator design

Figure 12: A tangible magnetic holder
With the same tasks tested in the user test one, second user
test was run with two subjects (Figure 10). interaction. The menus on the screen are divided into two
groups, main content and navigation. They are located
The two subjects were satisfied with using tangible different areas in order to help the users recognize these
magnetic holders instead of following GUI menus. menu groups easily.
However, subjects cited that the icon designs were not
intuitive. NEXT STEP
The interactions triggered by several tangible tokens and
Some observations during the user tests:
their own interactions are not vividly defined yet in this
• When the users were using tangible magnetic holders paper. Also, the lack of testing the users with a high-
together, many interactions occurred among TUIs. fidelity prototype is one of strong weaknesses of this paper.
• All users commented that using a TUI is much easier Therefore, the more studies should be reinforced to this
and faster than using a GUI, and very intuitive. paper in the near future.

FINAL DESIGN ACKNOWLEDGMENTS

Product design - refrigerator
The author of this paper is indebted to all people
The design keywords for this project were simple, user participating in this project, especially Professor Mike Scott
friendly, and easy of use. For these images, the basic colors and Niklas Anderson in Umeå University, Mall Allpere in
of this refrigerator are white and bright yellow green. The Myra Industrial Design AB, and the company Electrolux for
handle has half-transparent material for giving users a soft financial support.
mode (Figure 11). REFERENCES
1. Ishii, H. and Ullmer, B. Tangible Bits: Towards
Product design – a tangible magnetic holder Seamless Interfaces between People, Bits and Atoms.
This tangible magnetic holder (Figure 12) has a LED, so Proc. CHI1997, ACM Press (1997), 234-241.
users can notice whether this holder is activated or not.
2. DC Inside. Access at: http://www.dcinside.com
The size of this holder is large enough to be held and seen
by the elderly and children. 3. CNET Networks, Inc. Webshots. Access at :
http://www.webshots.com/g/
Graphic design - menus and icons
4. Patten, J. and Ishii, H. A comparison of Spatial
The GUI deign (Figure 9) shown in this paper is made for
Organization Strategies in Graphical and Tangible User
children and the teen, all menus and icons have its own
Interfaces. DARE2000, ACM Press (2000), 41-50.
graphic with name text to increase the users’ intuitive

32
 Sketching relations,
an interaction design perspective from research to design

David Werjefelt Patricia Panqueva

Institute of Design, Institute of Design,
Umeå university Umeå university
dadwet02@student.umu.se paapaa04@student.umu.se

ABSTRACT
DIMENSIONS OF INTERACTION
This paper has its grounding in that products of today can
In design, three dimensional form has long been reigning
be seen as increasingly dynamic, yet increasingly complex.
concept of dimension itself. But when designing dynamic
The components of a dynamic product have dimensions not
information technological products, other dimensions
only of visual quality; instead a pattern of sensory,
surface equally important. The dimensions can be collected
contextual and temporal relations forms the quality.
in three classes [Mike Stott, Personal communication,
Combining this statement with a perspective on interaction
2004] but are of course not new to our perceived world,
design as “relational sketching” we could grasp complexity.
rather unrewarded. First of, the visual dimension of a
By “sub sketching” on different dimensions and relating
product – where traditional industrial design has had as its
them with each other, a solution emphasizing all
reigning domain – is broaden to also include more sensorial
dimensions could be easier found. We propose also using
stimuli – most commonly auditory and tactile. Second, and
this method on gathered research, to illustrate the problem
perhaps of biggest importance, the area of interaction
where the design should starts. If relational sketching is
design gives great value to relation and context – both in
applied through the whole design process we believe users
terms of outer environment (e.g. relation to social life), but
would perceive the product as simple to handle.
also of inner context (e.g. relation among information).
Lastly, interaction is inevitable about events happening over
Keywords
Relational sketching, dimensions, relational perspective, time, what state follows which is very important feedback
sub sketching, natural relations. and adds a temporal relation in the structure of a system.
Mike Stott summarizes these thoughts in what he calls “the
INTRODUCTION three dimensions of form; sensorial, contextual, and
The diffusion of information technology seems ever to temporal”. It is important to recognize these dimensions,
continue. Interaction design is a discipline sprung from the and sub dimensions (e.g. auditory), as they expand the
traditional industrial design – a way to address and join in material that sketches of interactive artefacts are to deal
creating the flora of products and services which are far with.
from static, but increasingly interactive. Appealing to a
A RELATIONS PERSPECTIVE
broader range of dimension, primarily temporality and
contextually, it is important as interaction designer In order to discuss the method of relational sketching we
acknowledge that designing static products could different would like to point out how relations in and among these
from designing dynamic interactive ones. dimensions (temporal, cognitive, spatial, contextual, etc.)
combined depict the user’s needs and problems. A relation
is here denoted as a natural, or seemingly logical,
association between involved objects or states, which can
be interpreted as a connection between them. William
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are not
Gibson is attributed to having said [1], “…the future is
made or distributed for profit or commercial advantage and that copies bear already here. It's just not very evenly distributed.” which
this notice and the full citation on the first page. To copy otherwise, or can be seen as a statement of that what we perceive of the
republish, to post on servers or to redistribute to lists, requires prior permission. world is a pattern of relations. The future is merely built up
by enhancing some relations and suppressing others. This
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden points us to the task of designing – to find the relational

33
pattern of today, and modify it so that it better suits “the PRODUCING RELATIONS
needs tomorrow”. By now we want to stress how we see illustrating through
the whole design process, from research (finding relations),
SKETCHING PROBLEMS AS RELATIONS to brainstorming (mixing relations), designing (changing
Where does this bring us in question of “illustrating relations) and prototyping (testing relations).
interaction”? Well, if the user’s needs are a sum of
relations, we need to be able to express these relations. We
need to learn how to illustrate temporal relations, contextual
relations, etcetera, and combine them into useful
information. It can be seen as rendering knowledge out of
information.
In order to create solutions we need to modify the relations
creating the problem, and this starts already in the research
phase. By illustrating the relations found we’re actually
depicting the problem, from a designer’s point of view.
More over, since designing can be seen as rearranging, this
collected information can be used through the whole design
process, first as expressing a problem and later maybe
modified into a solution. Figure 1. Sketches (problem settings) and relations
(problem solving) between them.
SKETCHING AS THINKING
Sketching ideas can been seen as the designer’s most The process of illustrating in design has been described as
important tool, but stating sketching as a tool is maybe to “externalizing ideas (i.e. illustrating) and interpreting
weaken it’s position in design workflow. On the contrary of external representations as ideas” [3], but when explaining
a tool grabbed now and then, the method of illustrating, or design through the concept of illustrating relations, one
sketching, is at the very heart of design. It is a language and could use another emphasis. At a more pragmatic level it is
a dialog – the vehicle of thinking for designers, from of importance to notice how external ideas, illustrations, are
problem state to problem solution. The road to drive crosses continuously “put to test” [2] as an act of interpreting the
however itself many times, and it’s not a matter of going idea, but now in the light as a possible problem, i.e. a
straight on a one way street, from problem setting to problem state. This brings us to question where the first
problem solving [2]. problem setting is derived from. With the following
illustration (figure 2.) we want to attend how the
When working as a team it becomes even more apparent information retrieved from research, while depicted, can
how sketching is a vital part of designing. Sharing common form the basis of a first representation and problem state.
surfaces to present illustrations on expands the dialog to not
only between mind and illustration, but between minds.
Ideas are rephrased as the illustrated relations are
interpreted through the network of the cognitive relations of
participants.
If looking at the many dimensions of information that could
support the interaction design process, it becomes clear that
it is a complex matter transforming it into useful
knowledge. It has been noticed [Forbus & Usher, 2002] that
“in describing a complex artefact, one part of the sketch
might indicate how the overall artefact works, while other
parts of the sketch focus on specific details” and they refer
to the phenomenon as sub sketching. Figure 1. tries to
illustrate how each sub sketch (shapes in illustration) can be
seen as a problem setting and when solving a problem we Figure 2. Sketches as dialog, with research input.
create a relation (lines in illustration) from one state to
another.
When a relation is put to practice it can be seen as a logical
In order to extract further knowledge one needs to evaluate or illogical pathway in the flow of interaction and the goal
relations among these sketches and sub sketches. When of the designer could be seen as avoiding any seemingly
illustrating, this can be done by combining illustration (of illogical pathways in the final design. By starting to
relations) to form new relations by linking sketches or parts illustrate what is (an interpretation, still, of research data),
of sketches with each other.

34
5rather than what could be, we force the state of today to be A number of relations can be found in this illustration,
put to test. Is there a problem to begin with? between tasks, importance, complexity, cognitive load, and
the operator himself. It was a way to describe what was
While brainstorming, we attend the delight of creativity and
found, as neutral a possible, so that it could form the basis
that relations are not always easy grasp, nor all found
of problem statement. When research data is gathered, we
through logical reasoning. As this quote underline: “We
want can here emphasize that following phase of structuring
especially need imagination in science. – It’s not all
is not separate from visualizing, the build a co working
mathematics, nor logics, it’s somewhat beauty and poetry.”
unity.
[5] Creativity and imagination is very powerful means of
creating new relations and playing with old. Some
Brainstorming
brainstorming methods [Idea box, Creative…] even use This problem statement was then used to extract sub
cheer luck through “scrambling” create unthought-of of problems for brainstorming sessions. A number of
relations. brainstorming methods where used (them Idea box [6]), but
Designing in an iterative manner leads to a “self- they shared a focus on mixing relation pretty wild,
evaluating” loop of creating, changing, and testing relations sometimes all random, other times by expanding of each
over and over. It’s important to stress that the relations other ideas. In this way, you not only mix but also find new
created has only passed the cognitive architecture (i.e. unthought-of relations – a crucial ingredients in innovation.
mind) of the designer and the testing phase is therefore a
matter of running relations through a higher number of Designing
cognitive architectures (i.e. users mind). As we’ve pointed out the iterative method of design, we
here created, changed and tested, over and over, different
CASE relations that we thought interesting. Primarily we used the
The case here presented is from a project with forestry method of sub sketching on details, combining them and
machines. We where to redesign a screen interface testing relations from a more holistic viewpoint. Figure 4.
presenting information related to the process of harvesting shows a good example of where a structure of relations
trees, and the task was to reduce the time it takes to learn portrayed in research, interpreted as a problem, was used to
operating a harvester machine.

Research
Soon in the research phase we realised how complex the act
of harvesting was, and that we hade many loose elements of
no structure and poor relation to each other. The following
illustration (Figure 3.) depicts a mapping of different
relations we found when investigating harvesting.

Figure 4. Places or persons involved in harvesting

of trees, connected with a “memory key”

form a solution.
The first version of this illustration was a sketch where the
“memory key” (a personal key, containing information
relevant to the operator) was missing, and a problem of
information flow between places and persons was
discovered. By adding this key to centre of sketch a
solution surfaced, we added lines between the units and
now we could se the information “flow”.
Sharing common surfaces like white boards during the
whole project, from research to the design, made our
sharing and testing even deeper. It was a place where we

Figure 3. Felling of a tree.

35
could think together and share thoughts in a better way than The dilemma of having solutions without an actual problem
in an individual sketch. can be generally avoided. We emphasize on depicting
research information as illustrations not only for the
Final concept following design phase, but by doing so this information
The final concept in itself is not of importance for this would hopefully diffuse better in and between disciplines.
paper but we would like to use a portion of it to show a
relational of contextual and temporal dimension on DISCUSSION
information. Figure 5. shows a log that is currently being Relational sketching can be seen as the basis for continued
processed, cut. evaluation of how relational sketching is most efficient
applied in different part of design phases. What medias are
most effective in explaining temporal relations in the
research phase, or maybe temporal relations combined with
contextual. How can we sketch a prototype of tactile and
temporal relations, or how do we explain social connections
of an artifact?

REFERENCES
1. http://en.wikiquote.org/wiki/William_Gibson
Figure 5. Log to be cut. 2. Fällman, Daniel. In Romance with the materials of
mobile interactions. Department of informatics Umeå
University, Sweden.(2003)
Above, the quality of different parts of the log is marked
both in contrast and text on the illustrated log, a contextual 3. Stolterman, Erik. The design of information systems:
relation of this information. Further, this log (made Parti, Formats and Sketching. Informations Systems
construction and pulp parts together) is moving on screen, Journal, Vol. 9 (1999)
according to how the tree is moved by the machine outside, 4. Maria Mitchell, 1866 Quoted in "Maria Mitchell, Life,
a temporal relation. Letters, and Journals" by Phebe Mitchell Kendall,
http://research.dh.umu.se/dynamic/
CONCLUSION
Sketching in terms of relation is not a new thing since all 5. Michalko, Michael. Thinkertoys – a handbook of
sketches inevitable show relations. What we want to business creativity. Ten Speed Press, Australia.(1991)
address in this paper is more of a perspective on sketching 6. Forbus, K.D. Usher, J. Sketching for Knowledge
and interaction design. By keeping a constant use of Capture: A progress report. Qualitative Reasoning
sketches during the whole design process, and especially Group, Northwestern University. IUI’02, January 13-16,
join it with research phase, we believe more problems can 2002, San Francisco, California, USA.
be put to point, explained, and turned into feasible solution.

36
 Enhancing the User Experience of using an Electric Iron
Low Ko Wee
Umeå Institute of Design
Umeå University
SE-901 87 Umeå
Sweden
lowkowee@gmail.com
+46 73 692 4384
ABSTRACT worrying about ironing their precious clothes without
This paper presents how user testing studies, user getting any burn marks due to ignorance or inexperienced
observation and analysing the user experience flow of when using the electric iron. Some people even fear of
using an electric iron could be used to inform the design being scalded when using the electric iron. Another key
of next generation electric iron. The concept device was issue that the world has to contend with is an increasingly
designed to support and enhanced the psychological scarce energy supplies and natural resources. Re-cycling
aspect of the user and the environmental sustainability program has been implemented and promoted in most
aspect when using the electric iron. Psychological aspect countries. Energy saving houses and products that made
in this case means to let user gained self confidence and use of natural resources are used all over the world.
less worried about their safety when using the electric People are conscious about what types of things they buy
iron. Environmental sustainability means how wasted that might cause an imbalance of ecology on Earth. Such
heat energy of the iron could be renewed. This paper also items might include plastic bags that fail to break down
showed the design process for designing the user and cause pollution in the environment. It is not so much
interface and the industrial design of the electric iron and about improving the efficiency and functionality of the
how these two areas could be merged seamlessly using iron but rather to address on the psychological aspects of
the same design language, design elements and strategies. the user and the environmental sustainability of the
product. For this case, RE-energy, confidence and safety
Keywords are the areas this project is focused on.
Energy saving, confidence, safety, electric iron,
psychological, user experience RESEARCH
The research done in this paper aimed to accomplish two
INTRODUCTION objectives. The first objective was to research on
The world is filled with numerous brands of electric iron Electrolux product identity and how it could influence the
that are complicated to use. Various manufacturers all industrial design and user interface design. The second
over the world have been competing among themselves to objective was to gain a better understanding of the user
derive an idea of how the manufacturing cost of by using questionnaires, diary studies and conversation
producing electric iron could be reduced and thus helps to through web camera via msn messenger with the user and
lower the price to sell electric irons to the consumers. The user observation when using the electric iron.
manufacturers of electric iron would advertise and
compete among themselves by claiming the electric iron Electrolux Product Identity
to be of high wattage and power which has the capability To understand Electrolux product identity and design
to smooth out the clothes wrinkles. Some manufacturers philosophy, various Electrolux product brochures were
even claim that their electric iron emits more powerful collected and analysed. The images of Electrolux product
steam pressure than the rest of the manufacturers. The category was cut out and pasted onto a foam board to
labelling of the steam pressure is call PSI which means build up the mood feel for Electrolux products. (Fig. 1)
pressure per inch is so clearly seen in most of the electric
iron box. However, none of the ideas has focus on the
“users” point of view on developing a better iron that
makes ironing a better user experience. People are always

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
LEAVE
bear BLANK
this notice and theTHE LASTon2.5
full citation thecm
first(1”)
page.OF THEotherwise,
To copy LEFT
or republish, to post onON
COLUMN servers
THE or FIRST
to redistribute
PAGE to lists,
FORrequires
THE prior
permission.
COPYRIGHT NOTICE. Figure 1. Categorising Mood feel for
Electrolux product
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden

37
This mood board allowed me to capture and categorised Questionnaires were created with questions ranged from
the products according to various mood such as clean, daily activities of the user, psychological aspects of the
expressive and honest. The mood board allowed me to user, feelings associated with ironing, appliances the user
understand the visual and mental ergonomics of the used, pleasurable feelings when using appliances, likes,
industrial design and user interface design of Electrolux dislikes and lifestyle of the user. The questionnaires were
product. Some of the breakdown of industrial design then sent out to two females and one housewife from the
visual and behaviour language was icons, displays and two families. I had managed to identify most of the key
user interaction styles. In the kitchen product category: problem areas derived from the result of the
(Fig. 2) simplicity of form, straight lines, two colour questionnaires, photographs and online webcam interview
scheme with a feel towards classy style and the interplay with the users when using the electric iron and I aimed to
of materials finishes. The use of glass on top of the oven resolve the various problems and negative feelings people
controls suggest high-tech feel. In the house cleaning usually associated with it.
product category: (Fig. 3) subtle organic form with the
use of hard and soft form edge, two colour scheme with a
touch of bold colour. Bold colour for this case means
colour which was vibrant. A study visit was also made at
Electrolux networked store in Umeå, Sweden. This
visitation allowed me to interact with the various
categories of products thus gained a better understanding
of the look and feel of Electrolux and its subsidiary such
as AEG, Husqvarna and Electro Helios. Several rounds of
conversation with Electrolux industrial and interaction
designers further deepen my sense of understanding and
appreciation of Electrolux design philosophy which was
to provide user with pure functionality through simplicity
and design for ease of used.

Figure 4. User ironing clothes in the bedroom

Figure 2. Kitchen product category

Figure 3. House cleaning product category

Study of Home in Singapore

It was very important to understand the various issues
concerning with the lifestyle of the users living in the
house before designing a product as this may indirectly
affect the design concept. In this study, I am trying to
identify the key problem areas (Fig. 4) when the user was
ironing in the family. Two families were involved in this
study. Emails and photographs were used primarily for
this study. The photograph was based on the daily
activities done by the mother in the house. A plan view
(Fig. 5) of the house was sketch out and various
photographs were mapped out corresponding to the layout
of the room. The plan view allowed me to see the overall
view of the place and understanding of spaces which
affect human behaviour when using the iron. Several
photographs and conversation with the user revealed that Figure 5. Top view layout of a four room
user often moved the electric iron because of the apartment in Singapore
influence of entertainment such as television or the need
to chat with someone in order to kill boredom. The Ancient & Current Market Studies of Iron
location of the electrical socket affected the user mobility To understand the iron market, (Fig. 6) I started to do a
due to the constraint of the length of the electric cable. flow chart based on the development of the iron. Taking

38
note of which were some of the irons that very sellable
and why user decided to buy that particular iron instead
of the other. This study allowed me to understand the
entire historical development of iron which was crucial to
the design of the next generation of electric iron.

Figure 8. Mapping out the user experience

They were: 1) Warm up, 2) Wrinkled out, 3) Directional,
Figure 6. Flowchart of ancient and modern design 4) Circular. (Fig. 9)Through the user experience mapping,
of the electric iron one could understand how the user interacts with the
clothes, the iron and how the user shift the body weight
and body movement corresponding to the design of the
Once this was done, positioning chart was created to map ironing board and the types of shirt one is ironing.
out the segmentation of the electric iron in the market.

Ideal positioning
of the Electrolux
electric iron

Figure 9. Four Ways of ironing style

PROBLEM IDENTIFICATION
The following were some of the key problems identified
for the development of the electric iron.
Figure 7. Positioning chart of current electric iron • There was no proper location where the iron was kept
therefore the iron was always seen all over the place in
The positioning chart (Fig. 7) allowed me to gain a better the house. It visually disturbed the environment
understanding of electric iron segmentation that was because user associated it with stress and heat. The iron
either over saturated or unexplored. Through this always took up some time to cool down to room
positioning, I realised that manufacturers had always been temperature before the user kept it away. This wasted a
trying to compete against each other by selling ‘power’ lot of heat energy. User had to find their languages on
rather than innovation. Some examples of ‘power’ were the plastic panel in ordered to set the thermostat
to improve steam pressure (PSI). The higher the steam temperature. (Fig. 10)
pressure the faster the iron can smooth out the wrinkles –
according to the manufacturer.

Analysis of User
Another round of research methodology was employed to
garner more information from the user when using the Figure 10. Setting the thermostat temperature
electric iron. Some of the techniques used were \ based the types of clothes one was ironing. There
questionnaires, interviews and user observation. These were several language on the plastic panel
user studies were carried out in Singapore and Sweden.
The questionnaires were sent to eight users, either male or • Most of the clothes wore by the consumers were
female who are between the ages of 20 to 55 years old, manufactured worldwide. This created different
light to heavy user and from various occupations. clothes’ tag standard. (Fig. 11)

Mapping out the User Experience of Ironing

To gain an in-depth understanding of ironing and the user
experienced from the electric iron, the whole process of
ironing was done by two users. Video camera was used to
capture the process of ironing and the time taken to . Figure 11. Various manufacturers’ clothes tag
complete ironing each clothes was taken. The video was
then review several times to gain a better understanding THE CONCEPT
of the user. The videos were then analysed according to • To RE-use energy in the ‘Green’ home network.
the way user used the electric iron in a certain time
• To give assurance and confidence to the user when
period. (Fig. 8) The outcome of the result was that the
using the electric iron.
user adopted four ways of ironing style.

39
HOW IT WORKS CONCEPT EXPLORATION
This product consists of two parts: During the exploration phase, its main purpose was to
generate and brainstorm several user interface design and
• Electric iron industrial design concepts. The four main areas to explore
• Electric iron base unit were networked power line, manual and automatic
• Networked power line – An energy saving electric iron thermostat, thermal sensor and thermal indicator of the
which converts wasted heat energy to electrical energy electric iron. Three concept proposals were generated in
when it was cooling down and place onto the base unit the four main areas. The potential concept proposal was
[2] [5] [7]. The electrical energy was then shared then further developed based on the feedback from the
among appliances in the ‘Green’ home network via the user after paper prototype user testing. The mapping of
electrical power line. various functions was then carried out to define the best
location to place the user interface was. Three industrial
• Manual and automatic thermostat – The charged- design concept proposals were also generated to illustrate
coupled device (CCD) camera on the electric iron how the form could affect the function of the final design.
scanned the tag of the clothes [3]. The image captured
from the tag was then compared with a database of DESIGN DEVELOPMENT
graphical icons. Each icon was represented by different The concept was then further developed and to make it
temperature setting. Similar graphical icon control the more refined. Several sketches and foam prototypes were
optimum temperature needed by the clothes. built to test the design concept. Video camera was then
• Thermal sensor – The infra-ray thermal sensor in the used again to record how the user reacted to the design
electric iron continuously monitors the temperature of concept. Some improvement was made to the prototype
the clothes and will generate a sound if the user was after the feedback from the user. (Fig. 12)
going to over heat the clothes [4].
• Thermal indicator – A strip of dynamic material at the
edge of the electric iron reacted to the increased in
temperature of the electric iron and glowed into red
when it was hot and turned grey when it was at room
temperature [1] [6].

THE BENEFITS
The three areas that were beneficial to the users were that
wasted energy can be used by other appliances in the
home. It gave assurance and confidence to the user by
using the optimum temperature for ironing and the
improved ergonomics when ironing the clothes.

TECHNOLOGY USED
The following are some of the technology used for the
development of the electrical iron. Figure 12. Prototype testing
• Charge-couple device (CCD) array and cold cathode
DESIGN REFINEMENT
fluorescent lamp (CCFL) – To capture graphical icons Once the final changes had been defined, Macromedia
on the clothes tag. Flash (A vector and raster based 2D animation software
• Image matching software application – To compare packages) and Alias StudioTools (A 3D CAID –
similar graphical icons and set the optimum Computer Aided Industrial Design) software were used to
temperature for the clothes that one is ironing. create the final physical appearance model and rendering
• Infra-ray thermal sensor – Using infra-ray to monitor done to visualise the concept.
the temperature of the clothes.
Product Design
• Energy saving microchip – To convert thermal energy This two-directional universal design (Fig. 12) allows
to electrical energy to be used by the networked home. user to have the freedom to inter use both of their hands
• Carbon ultra capacitor – To store electrical energy to when ironing without worrying about the orientation of
be used by the networked home. the iron. It also speeds up the ironing process through its
streamline design. With the two-sided electric iron, user
USER SCENARIO OF USING THE ELECTRIC IRON does not need to turn the iron around to smooth out the
Once the design concept was derived, the next step was to crease on the clothes. The design element chosen is clear,
map out the scenario visually using drawing and writing simple and straightforward reflecting the design
as a tool to illustrate the idea explicitly. This served as a philosophy of Electrolux.
way to map out the user interface required at various
stages of the electric iron.

40
 CONCLUSION
The design concept presented in this paper offered an
indication of how the product interaction might be
designed using the findings from the user feedback and
user observation alongside various established
prototyping techniques. By paying particular attention to
different stages of user experienced the author could help
to refine and define the experienced that was crucial for
making the electric iron enjoyable to use.
Figure 12. Final product design
ACKNOWLEDGMENTS
User Interface Design The author would like to thank those people who
The buttons of the thermostat design (Fig. 13) were participated in this study for their time, enthusiasm and
designed to encircle the display button. This allowed the invaluable insight. Especially to Professor Mike Stott and
user to view the heating modes clearly from the feedback Niklas Andersson from Umeå Institute of Design, Umeå
of the buttons lights and display. The ‘A’ on the display University, Mall Allpere from Nokia Mobile Phones,
means that the thermostat is in automatic mode and ‘M’ Britta Burlin and Mathias Tingstrom from Electrolux who
means that the electric iron was in manual mode. The is sponsoring and tutoring the author during this project.
reason for automatic and manual heating mode was that
this electric iron was targeted at inexperienced users to REFERENCES
proficient users in ironing. Manual mode allowed the user 1. About The Physics Hypertext Book.
to over ride the heating control as some user thinks that http://hypertextbook.com/facts/2002/SaiLee.shtml
one can iron a clothes faster if it was hotter. 2. Combine Heat And Power Capturing Wasted Energy.
http://www.aceee.org/pubs/ie983.htm
3. How Stuff Works.
http://www.electronics.howstuffworks.com/digital-
camera3.htm
4. Mitsubishi Electric Research Laboratories.
http://www.merl.com/projects/UrbanMatch
5. NewScientist.com: Microchip Can Turn Heat Into
Figure 13. Thermostat control Electricity.
http://newscientist.com/news/news.jsp?id=ns9999186
The graphical icons used on the buttons and labelling 2
throughout are simple and clean which is in accord with
the visual design strategy outline in the early phase of the 6. Dynamic Material Library.
project. (Fig. 14) http://research.dh.umu.se/dynamic
7. Nitride Nanostructures For Thermionic Energy
Conversion.
http://engineering.purdue.edu/MSE/Fac_Staff?Faculty
/TURNER_LAB/Nitride.

Figure 14. Graphical icon design.

41
 Information Graphics as an active tool during the
Interaction Design process
Santiago Barriga Lisa Hultgren
Institute of Design Institute of Design
Umeå University Umeå University
barrigas@hotmail.com hultgren.lisa@telia.com
+46 73 8125007 + 46 70 7264737
ABSTRACT to identify opportunities to develop design solutions or
This paper presents a graphic structure used as a tool to improvements, the interaction designer has many options to
organize, illustrate and handle information during the early manage information in a practical and efficient way.
stages of the interaction design process. This seems to be an
alternative way to deal with today’s need to manage large The use of lists and diagrams is frequent during design
amount of diverse data making it visually available for the projects. Information, collected initially as notes from
designer. The development of this graphic method is observation or interviews can be easily translated into a list
illustrated with a case where the tool was applied as well as of sequential events.
a short guide to the use of the tool. The results demonstrate It is important to describe the situation in detail. In
the benefits derived from the application of the method. interaction design is very popular de use of the hierarchical
task analysis method (HTA) [Annett and Duncan, 1967].
Author Keywords Originally developed to identify training needs, HTA
Information Handling. Information Graphics. Interaction consists is used to describe a user’s goal or task by dividing
Design. Design Methods. it into the steps needed to complete it. Then each step is
divided into sub-steps and so on (Fig. 1). Then tasks are put
INTRODUCTION together as a plan in order to describe a situation. HTA can
All the events and reflections in this paper are based on the be represented in a graphical way and visualized as a whole
experience of an academic interaction design project. The for the first time giving the possibility to include comments
focus was the improvement of the control system of a and observations.
specialized forestry vehicle in terms of user interface. The
goal was increasing profit from the activity by reducing the Task
effects of cognitive overload on the workforce’s health,
performance and safety.
During the early stages of the interaction design process Step 1 Step 2 Step 3 Step 4
data gathering takes places. This serves to the purpose of
approaching and understanding the elements, context and
factors involved in the situation.
Step 2:1 Step 2:2 Step 2:3 Step 2:4 Step 2:5
The diverse media in which information is registered such
as texts, sketches, photographs, sound recordings and Figure 1. A graphical representation of HTA. Adapted from:
videos can be collected sometimes in large quantities; this, Preece, J., Rogers, Y., Sharp, H. Interaction Design. John
in addition to the highly detailed information resulting from Wiley & Sons. New York. NY, USA, 2002, 231-232.
initial data analysis, can become problematic for the
researcher if not properly organized and mapped. In order PURPOSE
to get a detailed overview of the studied situation that leads The use of the HTA method helps to build a basic structure,
a time line that keeps track of the sequential tasks the user
is performing but yet not intended to accomplish the goal of
showing or describing what is happening to him at a
cognitive level on a certain moment. To make this possible
Permission to make digital or hard copies of all or part of this work for there were additions made to the graphical representation of
personal or classroom use is granted without fee provided that copies are not the initial HTA in order to it make descriptive of the
made or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. To copy otherwise, or
situation at a cognitive level. The purpose of this dynamic
republish, to post on servers or to redistribute to lists, requires prior permission. graphic method is setting an adequate scenario for the
identification of the particular mental processes taking
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden place, their frequency, importance and eventual

42
disturbances in order to detect the actual causes of cognitive
overload during the operation sequence.

CASE
Information was collected in diverse ways and media.
Initially at lectures on Sweden’s forestry economy, history
and workforce labor and cognitive conditions; later, during
presentations on the machinery and computer system,
technical information was provided; harvester training
simulator trial and visits to the forestry machine assembly
factory and the woods where forestry works using this
equipment were taking place. Notes and sketches were
collected both by hand and computer. Images were
recorded on video tape and digital cameras. The Figure 2
information in electronic media (photographs, videos, word Figure 3 is a “zoom” into the part of the sequence found to
processing, graphs) was reviewed, edited, printed out, be the most critical during the cutting process and the part
annotated and integrated to the rest of the information, we decided to focus on in our project. Compared to other
arranged and mapped. moments during the work day, this sequence involved
Data was gathered to illustrate the driver of a forest diverse simultaneous physical actions and cognitive
harvester machine’s working situation, the tasks he has to processes. Then iterated many times.
do, what kind of information he is exposed to and what kind
of decisions he has to make in different situations to detect
the most critical moments of cognitive overload, one of the
main causes of decreasing productiveness in the forestry
business, and come up with a design solution. Graphs
developed during the descriptive phase helped to map the
information as expected and the one resulting from analysis
became the expected active tool.
Instead of the usual “tree” structure normally applied to
graphical representations of the HTA method, a linear,
sequential one was used. The decision was taken in order to
represent time progress in a more appropriate way.
Subdivisions of the particular tasks are represented then as
Figure 3
text lists in the lower part of the graph.
Then we looked at the information on the screen that the
The original size of the following illustrations cannot be
driver is exposed for during his work (Fig. 4). We tried to
reproduced in this format so texts are not legible. Even
visualize how big the cognitive load is today and as a result
though, it serves to describe different stages of the same
from the previous graphics we could now make suggestion
graphic growing by addition of elements in a proper way.
on how to lower the cognitive load. This both show
Figure 2 shows the whole working day with an iteration of information showed during the sequence of the
the actual cutting sequence. The blocks added above shows actions/tasks.
information about different factors that affect the actual
sequence. This information is collected from video film and
notes from observation, interviews, facts from lectures and
the company, and so on.

Figure 4

43
RESULTING METHOD
The experience that began with the graphics developed in a
Cognitive process Work space Context
natural, spontaneous way evolved in a set of two graphic
structures.

Putting it all together: the descriptive phase

Establishing the sequence FACTORS
The first set of graphics is made to map all the information
collected to obtain a detailed overview of the situation
beginning with the graphic representation of the initial
hierarchical task (functional) analysis with the description
obtained from observation and preliminary studies (Fig. 5). Action/task 1 Action/task 2 Action/task 3 Action/task 4

Subtask 1:1 Subtask 2:1 Subtask 3:1 Subtask 4:1

Subtask 1:2 Subtask 2:2 Subtask 3:2 Subtask 4:2
Subtask 1:3 Subtask 2:3 Subtask 3:3 Subtask 4:3
Action/task 1 Action/task 2 Action/task 3 Action/task 4
Figure 7
Subtask 1:1 Subtask 2:1 Subtask 3:1 Subtask 4:1
Subtask 1:2 Subtask 2:2 Subtask 3:2 Subtask 4:2
Subtask 1:3 Subtask 2:3 Subtask 3:3 Subtask 4:3
Making use of information: Analytical phase
Figure 5 The designer then decides where to take action and focuses
on a particular action or routine. By representing when in
Context description time a particular event takes place, the information defined
This is complemented with contextual information that during the last part of the descriptive phase is refined.
describes the situation in terms of the surrounding space,
objects and operational conditions of the user to identify the At this point, the map should provide enough information to
different kinds or categories of external elements directly define order and hierarchy of the steps required to
involved in the situation (Fig. 6). accomplish the task so the designer can decide the
appearance and temporariness of graphic elements on the
redesigned user interface depending on the information
FACTORS needs of the user.
The linear structure of the graph has evolved from a linear
one (Figs. 6-7) to a two axis chart where actions and
cognitive processes are on the x axis (Fig. 8). The y axis
contains information shown the actual user interface.
Action/task 1 Action/task 2 Action/task 3 Action/task 4

Subtask 1:1 Subtask 2:1 Subtask 3:1 Subtask 4:1

The figure shows how a sequence involving a large amount
Subtask 1:2
Subtask 1:3
Subtask 2:2
Subtask 2:3
Subtask 3:2
Subtask 3:3
Subtask 4:2
Subtask 4:3 of simultaneous actions and decisions at different levels
generating textured zones in the graphic, where the most
Figure 6
dense represent higher levels of cognitive overload (light
fields), can be simplified by making some of them
Critical moments
intermittent or just invisible until needed (dark thick lines).
These categories of elements appear initially as titles. As
details are needed they can be included as comments or Displayed
information Action 1 Action 2 Action 3 Action 4

sub-categories. Most of the contextual information obtained A

B
by direct observation was complemented and detailed after C
Displayed information

D Information that is necessary

an exhaustive review of photographs and video recordings E Information as“pop-up”

(Fig 7). “Critical moments” of cognitive overload are F

G
defined at this point as the ones with more factors involved H

simultaneously in the situation at one particular instance of

Figure 8. The word “action” on this table refers to physical
the sequence. observable actions as well as to cognitive processes.
First hand information on cognitive issues was collected in-
situ with the user. He provided information of his An active tool
experience in this kind of work, personal opinions on the The physical actions mapped on the descriptive graph were
situation, a general description of the task sequence and found to be equivalent to those represented on the GUI of
details on his cognitive processes that was included in the the computer system in the tree cutting machine. This
map in the form of hand written annotations. makes possible to apply any finding or design criteria,
derived from the user–context study to the mediating
computer system timeline.

44
The “active” nature of the analytical graph appears when, keep all the information visible and available all the time
only by showing the light fields serves as a template where reducing mobility and archiving possibilities to some
the designer can explore as many timeline alternatives, with extent.
their respective sets of design criteria or parameters, as they • As a suggestion for the administration of the material
decide. collected at the beginning of the project, keeping a log
book or annotating date, time, source and other reference
CONCLUSIONS
help to reorganize the pieces in case they have been
The modified hierarchical task analysis graphic, not only
moved. It can also facilitate the construction of the initial
proved to be simple and effective but flexible enough to
sequence.
admit additions extending its well-known capabilities to
manage the description of physical actions, cognitive RECOMMENDATIONS
processes and time progress. Graphics helped us to structure and visualize the
The analytical graph was successful at representing a group information we collected, and later on helped us make
of simultaneous events causing cognitive overload. It also decisions and present our ideas. Information graphics were
serves as an active sketching format during the early applied on the whole design process, not only made for
creative stages during the design of a graphic user interface. presentation but also for analysis and decision making. It
helped us keep our goal.
This way of using and collecting information can be
generalized to other design disciplines projects because More interaction designers should try using Information
,although some times cognitive processes or time are not Graphic during their design process as an active tool not
considered while developing solutions, putting attention on only using existing methods but making adaptations of
these aspects may give a more detailed and accurate them according to the actual needs of their projects, giving
overview of the evaluated situation, and as a consequence, a try to the benefits of the potential of information graphics
more material for reflection and a conscious creative on creativity and data management.
process.
ACKNOWLEDGMENTS
DISCUSSIONS For their guidance and attention we would like Mike Stott,
The application of this method in different fields and Nicklas Andersson and Carin Larsson at Institute of Design
situations can be left open for discussion since it has been – Umeå University, Iwan Wästerlund at SLU, Bengt-Arne
tested just in one single case. We encourage our colleagues Wallden at Komatsu Forest and anybody directly or
and anybody interested to try this method on their own indirectly involved in this project.
particular specialties.
The following can be points of discussion, suggestions and REFERENCES
other observations. 1. Annett, J., Duncan, K.D. Task analysis and training
design, Occupational Psychology, 41, 211-21.
• This method was tested on a case that had a predefined 2. Norman, D. Things That Make Us Smart. Basic Books.
need or situation to solve. To prove it could be used as a New York. NY, USA, 1993.
generic method for interaction design projects, it would
have to be tested because might not work in the case we 3. Preece, J., Rogers, Y., Sharp, H. Interaction Design.
also need to identify needs from scratch. John Wiley & Sons. New York. NY, USA, 2002, 74-89,
231-232.
• The most important limitation noticed as the descriptive
graph was completed was that using a large format with
big enough text and graphic elements was the only way to

The columns on the last page should be of approximately equal length.

45
 Interaction Designers Collaborating in Matching the
Physical and Virtual Interface in Libraries

Linda Bogren Madlene Lindström

Umeå Institute of Design Umeå Institute of Design
Umeå, Sweden Umeå, Sweden
liabon99@student.umu.se maelim00@student.umu.se

ABSTRACT BACKGROUND
In this paper, we illustrate the design process of two col- Our aim as interaction designers is to create a platform for
laborating interaction designers through an educational the user. To acquire the necessary knowledge we have to
project. We discuss the importance of an ethnographic ap- use different methods and tools in different situations. The
proach and multiple testing of ideas to inform the design society and thus the user are under constant change and the
process, allowing us to create a platform for the user. In the designer must have an understanding of this and continu-
paper we talk about that it is essential to use methods and ously create new methods and tools to understand the user
tools according to the task and situation. We show how the and the environment. We have an ethnographic approach to
use of metaphors increased the usability of a concept and our work in the sense that we use ethnographic field study
also helped brainstorming to get new ideas. The concept to inform the design. We need to be clear about the fact
solution is designed to support the visitors’ and the librari- that we are not ethnographers nor do we possess all the
ans’ behaviour in the library through a system containing tools of an ethnographer.
maps, compass and colour coding. Ethnographers Alex Taylor and Richard Harper have at-
Keywords tempted to use field studies to inform design, using field-
Ethnographic approach, low fidelity, multiple testing, work and qualitative analysis to produce design sugges-
teamwork, platform, interaction design, metaphors tions, in cooperation with interaction designer Sara Berg.
They argue that more systematic methods for uncovering
INTRODUCTION
and using such practices to inform design should be inves-
The framework for the project described in this paper was a
tigated. [1] We support this argument. However, various
five-week project called The Library As Interface, carried
criticisms about the ethnographic field studies that aim to
out in cooperation with the Umeå City Library and Umeå
inform design have been levelled. Anthropologist Diana E.
municipality. The aim was to match the library interior
Forsythe mentions that some ethnographic studies have
with the PC-based search engine interface. Also to match
focused on the wrong things and thereby missed or misin-
the professional tools of the librarian to suit the visitor at
terpreted information, producing too vague design re-
the library. Our target group consists of two types of visi-
quirements. [2] To us, this discussion about ethnography
tors. Firstly, is the stroller who is looking for inspiration
and design shows that there is reason to keep an ongoing
about what to read, with an open mind. Secondly, the struc-
debate and to investigate systematic methods that can be
tured visitor who knows what he wants immediately.
useful. It is important that we as designer do not claim to
We used metaphors to generate ideas that resulted in dif- be ethnographers but rather that we find the tools and the
ferent concepts and tests. Through many iterations of the knowledge of ethnographers useful to us.
tests we came to conclusions about the final design.
TEAMWORK
Interaction Design is a quite new field of design and is in- In our previous studies teamwork has been carried out ex-
fluenced by many other areas such as traditional product clusively in the initial research phase. In this project the
design, computer engineering, graphic design, ethnogra- two of us wanted to carry the teamwork all the way
phy, sociology etc. The influence consists of making use through, to be able to compare the experience of working
of the knowledge of these fields and also borrowing meth- alone and in collaboration. We also wanted to come a bit
ods and tools. closer to reality where you do not work as a solitaire and
Permission to make digital or hard copies of all or part of this work for you have to argue your case and compromise. It was as a
personal or classroom use is granted without fee provided that copies are not creative process where we would constantly build on each
made or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. To copy otherwise, or
other’s ideas. One of the things we found out is that the
republish, to post on servers or to redistribute to lists, requires prior permission. ownership of an idea seizes to exist in teamwork. However,
to be able to carry out a fruitful project it requires a func-
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden

46
tional group constellation. In this project we felt that we correct connection between the physical and the repre-
came further by collaborating instead of working individu- sented world, they have to adjust the physical items to the
ally. 2D items on the map. By utilizing a compass, a north-south
correction of the map is possible.
INVESTIGATING AND DEVELOPING THE PHYSICAL
AND VIRTUAL SPACE
We created a scenario based on observations to document
how the search system in the library works and how it ap-
plies to the physical room. A scenario is a concise descrip-
tion of a persona using a product to achieve a goal. It high-
lights the difficulties in a product or product interface and
it shows what a person is doing and in what sequence. A
persona is a simple, effective and a powerful tool to de-
velop a description of the user and what he wishes to ac-
complish [3]. Another tool to help verbalize the problems is
to the cherry split method, which we used to get a different Figure1. First test session; Symbol Recognition and Pattern
perspective of the problems listed. [4] Recognition
FIRST TEST SESSION
When we knew what and how things were done we wanted Symbol Recognition
to know why and in what context. By observing and ana- In the test Symbol recognition the idea was to connect a
lysing the visitor’s behaviour in the library, we tried to get symbol on an abstracted map with a similar symbol in the
a feeling of the atmosphere in the different sections of the room to get an understanding of the room (Figure1). Our
library. We talked to visitors at the library and interviewed idea was that if the same task could be solved in the same
the librarian on different occasions to get an understanding time using small or great means, it would be preferred to
of the core values of the library to the visitor and the librar- use small means. The less detailed the map can be the bet-
ian. ter to allow other kinds of information, such as the book
Our approach was to start our work with the physical space title etc.
of the library to be able to connect it to the virtual space. We set up the test using low fidelity prints on paper. The
We looked at possible ways to translate the classification parts that constituted the test were the paper symbols in the
system, to also suit the visitor. The sign system in the li- floor, put out in five different locations, a printed map with
brary is limited by the possible viewing angels and did not the symbol on one side. The abstracted map showed the
show the borders of the sections. We wanted to make this main characters of the walls and the location of the user
division of sections more distinct by looking into different and the object that was supposed to be located. The symbol
ways of colour coding the sections and other kinds of sign was a filled circle cut in half, one half on the floor and the
systems. The virtual space of the library, the library cata- other half scaled down on the map, addressing the Orien-
logue, gives no indication of how the physical room looks teering metaphor. We wanted to see how the users behaved
like. We looked into how to connect these two to give the when matching these two together. We let three users of
user a better understanding of the room. various age and gender perform the test.
METAPHORS The tests made us see things that we had to develop further.
We used metaphors as a catalyst to generate ideas. [5] We All users understood that they could connect the symbol in
found IKEA, Orienteering and Subway suitable, since these the map and on the floor to get some kind of direction, like
are metaphors where the aim is to get to a specific place or a compass. However, it was not clear to everyone that the
to find something. The metaphors resulted in three con- symbols should form a circle. The symbol had to be devel-
cepts that we in the next step tested parts from. We looked oped to make sure that there would be no misconception on
at IKEA as a way of finding what you are looking for by how to connect them. We also had to develop the symbol
following a guided path through the sections of the store. to make it feel more public since there was a conception
Some items are found in the warehouse by using a numeri- amongst the users that it was personalized.
cal and alphabetical code. We looked at the Subway as get-
Pattern Recognition
ting from one place to another by a limited path, consti- In the test Pattern recognition we wanted to see how peo-
tuted by the different routes. There is a simplified map, ple read and understood 2D images and how they would
only useful in this specific setting, which uses color coding recognize and understand patterns, addressing the Orien-
to clarify different routes. Markers in the physical world teering metaphor. We used the top view of the layout of the
inform the people where the subway is. We looked at the bookcases in the library and printed it on A3 paper. We had
Orienteering as a free path where the objects in the physi- copied and printed sections of the map in scale 1:1. With-
cal world are marked out on a one dimensional map, ena- out instructions the users were given these sections of the
bling the user to find their own way to the goal. To make a big map one by one and we wanted to see how they would

47
use them. We tested three levels of abstraction. The first symbol on the card representing the compass was a bit too
with only the bookcases marked out, the second we added vague so we saw the need to develop the clarity of the
markers in greyscale and the third colored markers. We graphics.
wanted to see how much information they needed to per-
Pattern Recognition for Guidance
form the task. This one-dimensional map was a section copied from the
When given the most abstracted test without the markers, big map in the same scale. Compared to the former test this
the test persons instinctively began to make a puzzle of the does not show an overview of the library but only a sec-
sections by matching the sections with the big map. When tion. We made it in black and white to resemble a receipt.
introduced to the same map but with markers they thought This contained information about the location of the book,
it was even more direct, the one with the colored markers author and title. The receipt map showed the section of the
was the most direct of the three. We came to the conclusion library where they could find the book. The compass sym-
that the users did manage to recognize patterns and match bols were represented as markers in the receipt map. The
them up easily and with an added element it was even more compass symbols in the floor then acted as markers in the
direct and fast. physical world, addressing the Subway metaphor as well as
Orienteering.
When we introduced the users to the test we pretended that
the search in the catalogue was already made and that they
should find the selected book in the library. Our users were
two girls between 20 and 30 years of age.
Instinctively they connected the receipt map with the big
map and made a puzzle to get the direction. They got con-
fused once about their whereabouts and one of the girls
Figure2. Second test session; Pattern Recognition for used the compass symbols in the floor to match with the
Guidance and Symbol Recognition for Guidance compass symbols on her map and found the right direction
SECOND TEST SESSION again. The conclusion from the second test session was that
The aim with the test was to discover how a user would both concepts would work equally well in reality.
interpret the one-dimensional map to the three dimensional
objects in the library (Figure2). Common denominators for
the tests were a printed map of the layout of the library
showing the sections as colour coded. The bookcases were
what defined the library on the map. The map showed the
position of the user. There were also compass symbols
marked out in the map and placed on the floor throughout
the library. The compass symbols were colour coded ac-
cording to the sections they were in. Both tests derive from
the metaphor of Orienteering where the objects in the real
world have to be defined on a simplified map of the reality.
Symbol Recognition for Guidance
The one-dimensional map in this case was printed on a
small piece of paper resembling a library card and was a
Figure3. Final result; Compass Card-Symbol Recognition
representation of the entire library, identical to the big map.
for Guidance in Libraries
A symbol representing the compass was printed on the
card. The card map contained information about where the FINAL DESIGN: Compass Card- Symbol Recognition for
book was located and the name of the author and title of Guidance in Libraries
the book. Due to the fact that the concept we tested in Pattern recog-
When we introduced the users to the test we pretended that nition for guidance would require some further develop-
the search in the catalogue was already made and that they ment concerning the markers for instance, we chose to pre-
should find the selected book in the library. Our users were sent the concept we tested in Symbol recognition for guid-
a couple in the age of thirty. ance as our final concept, since that is more developed at
this stage.
They instantly started to adjust the card map to the big map
to get the right direction. They used the card map during The physical space – the library
the search to find the book and they succeeded within The library was divided into six sections; fiction, technical
about 2 minutes. They noticed the compass on the floor but literature, media, child and youth, magazines and reference
didn’t have any need for it. They felt that the graphical

48
literature. Each section is represented by a unique colour again, much like orienteering. Also by noticing the colour
that shows in both the virtual and physical room. changes when moving through the room they will get a
We understood that the interface for the user and the librar- sense of location and direction.
ian needed to have different information and value and that When borrowing the book they insert the card into the ma-
is why we separated these. We chose to keep the letters chine and the ink that was printed with the information will
used today to represent different categories in the library, have disappeared. This makes it possible to use over and
since we wanted to be able to keep the librarians work tool. over again. Example of existing technology to support
Also, letters are universal and can be understood by all these actions is Toshiba’s e-Blue system that prints blue
nationalities. On the signs in the bookcases we have two ink and then removes it. [6]
separated fields of information; one is directed to the librar-
CONCLUSIONS
ian consisting of the existing classification system and the The teamwork was as a creative process where we would
other is directed to the visitor. This information is numbers, constantly build on each other’s ideas. The ownership of an
which is also a universal code, so the information the visi- idea had to seize to exist in the teamwork, to keep the ob-
tor would get about where to retrieve a book would be for jectivity the participants. To be able to carry out a fruitful
example Q4, while the information to the librarian about project it required that the team would keep an open and
the same book would be Qgb. This numerical and alpha- honest dialogue, which can be quite demanding from time
betical code also refers to the IKEA metaphor. to time. In this project we felt that we came further by col-
The colour coding of each section is meant to give orienta- laborating than we would have if we worked alone. One
tion in the physical room. About the visitors’ position, reason for this was that the teamwork required more struc-
where to find the book and through which sections they are turing and constant reviews of the work which lead to a
moving. The colours are shown on the sides of the book- better overview.
cases, the signs in the ceiling, in the bookcases and on the The use of metaphors worked well as a tool for generating
compass in the floor. The color coding derives from the ideas and increased the usability of the final product. It was
Subway metaphor, by differentiating sections. The shape of also an appropriate base to start from in our teamwork
the signs containing the letters is triangular since that gives since we both knew about the metaphors and could discuss
more visibility. There are compasses in the floor through- around them and our own experiences of them. It quickly
out the entire library and their goal is to give direction formed a common ground for us to stand on.
since they are all oriented in north-south direction and fol-
low the colour code of the section they are in. This is con- The multiple testing on different levels of abstraction were
necting to the compass in the Orienteering metaphor. very useful and made us aware that sometimes it is not nec-
essary to test the entire concept but selected parts of it. We
The virtual space – the library catalogue found it important to be aware of what you are testing and
The GUI (Graphical User Interface) used on the computers why. You also need to be aware of the fact that it can seem
today is made by the company Axiell. By using the new confusing to the test persons, since they do not see the big
version of Axiell’s GUI as a foundation, we illustrate how picture. Hence, it is important to know how to inform the
easily a connection between the virtual and physical space test person about the situation.
can be done by adding the colour coding system.
REFERENCES
The connection between the virtual and physical – the 1. Berg, S., Taylor, A.S., Harper, R. Mobile Phones for the
compass card
Next Generation: Device Designs for Teenagers. Pro-
By the workstations, in front of the visitor, there is a 2D
ceedings of CHI ’03volume.5, Issue.1. (2003), 433-440.
map of the layout of the bookcases in the library. Each sec-
tion is marked with the designated colour. The map ad- 2. Forsythe, D.E. “It’s Just a Matter of Common Sense”:
dresses the Orienteering metaphor. Ethnography as Invisible Work. Computer Supported Co-
operative Work 8, Kluwer Academic Publishers, (1999),
The same map but in smaller format is printed in colour on
127-145.
the library card with an integrated compass. The user has
the option to insert the library card into a small printer by 3. Cooper, A. The Inmates are Running the Asylum. H.W
the computer to get the information about the location of Sams, US, (1999), ch 9.
the book printed on the card. If they feel the need to they 4.CherrySplit,www.mycoted.com/creativity/techniques/che
can also choose to get a paper print out on the author and rry-split.php. 2004-12-22
title of the book.
5.Svanaes, D., Verplank, W. In Search of Metaphors for
Before going off to find the book they adjust their card map Tangible User Interfaces. Proceedings of DARE ´00 (2000)
to the big map to get the right direction. If they loose direc-
6. Toshiba's “e-blue(TM)” Decolorable Toner and Erasing
tion of the card map during the search in the library they
Machine Set to Boost Paper Recycling at Offices in Japan
can use the compasses on the floor by matching them with
www.toshiba.co.jp/about/press/2003_12/pr0201.htm,
the card compass and thereby getting the right direction
2003-12-02

49
 Using Phonological Strings as a metaphor to connect
Auditive, Graphical and Tactile User Interfaces

Ida Ristner German Leon

Umeå Institute of Design Umeå Institute of Design
Umeå University, Sweden Umeå University, Sweden
ida_ristner@hotmail.com germandleono@yahoo.com
ABSTRACT design process could consider different user interfaces and
The purpose of this paper is to show the findings of a structure to make the learning process faster and easier to
seven-week study case: “Information handling in new users.
specialized vehicles”.
The world we all live in today focuses on the design of
The human brain’s ability to process information input visual user interfaces. The way technology is going from
through the different senses (vision, hearing, touching and virtual technology to tangible objects, makes it necessary to
feeling), help to understand how better designed user analyze other user interfaces to reduce the amount of visual
interfaces can reduce the overload of information, which is noise (non-relevant information) and the number of
a problem. procedures to memorize.
The phonological strings theory [12] (different key sounds Cognitive Workload
put together by the brain to create a complete picture) has The amount of information has become a real problem in
been used as a metaphor in this project. It helped to design today’s society. In the field of specialized vehicles, we
a connection between different user interfaces and to found that the load of information, to operate and process,
generate better associations. Combining graphical, auditive is so heavy that users need almost two years to learn the
and tactile user interfaces and structured information makes process.
it possible to reduce the cognitive workload.
As a result important issues like being productive, receiving
The results suggest a need for deeper research into new visual, audio and tactile information separately, cause
ways of linking man and machine. problems to experienced and new users.
Author Keywords In general terms, the problem affecting this industry, is
TUI, GUI, AUI, Phonological Strings, Connection, shared by other industries and by individuals. The reason
Associations, Structure, Senses and Cognitive Workload. for this is that the era we live in today offers too many
INTRODUCTION possibilities to people in general. This makes learning hard
At the beginning of the information technology era, and complex. Just think of all the new mass produced
information was the key to success. The problem now is to artifacts. When will we have time to learn how to use them?
deal with the massive amount of it. How can we sort out the There is a need to improve and understand the information
unnecessary information and find the important information technology because of the demand for fast and productive
in the context? Today we are “forced” to handle and accept processes. The key to all these different aspects is in the
a large amount of information, which might prolong the way we manage information trough our brains. As
learning process in different working areas. designers we can contribute, by reducing the amount of
This paper presents a humanistic approach to how the information that has to be processed and memorized,
helping people to understand how things are used thus
reducing their need to memorize.

Permission to make digital or hard copies of all or part of this work for
What are GUI, TUI and AUI?
personal or classroom use is granted without fee provided that copies are not User interfaces are the communication elements used to
made or distributed for profit or commercial advantage and that copies bear operate objects, better known as the link between man and
this notice and the full citation on the first page. To copy otherwise, or machine.
republish, to post on servers or to redistribute to lists, requires prior permission.
This paper will focus on three user interfaces, graphical
UIDSC´05, January 13, 2005, Umeå Institute of Design, Sweden (GUI), tactile (TUI) and, auditive (AUI).

50
These interfaces are the extension of the human senses small dots represent the different options the user has when
when operating the vehicle, but senses have different ways taking a decision, and the orange represents the external
of operate and use specific types of interfaces, which can output that is generated by outside factors such as noise.
generate a large amount of information making it The diagram also shows three areas circle in gray which are
complicated to memorize routines and to interact. the three interfaces that are analyzed in this project.
Integrating GUI’s, TUI’s and AUI’s, with improved
understanding of the short-term, long-term and sensor
memory of the object interface, will generate a reduction of
the amount of information to be stored in the brain, and
could speed up the interaction of the interfaces.
METHODS
During this research these methods where use: Interviews,
Field Research, Analysis, Ideation, and “Low fidelity”
prototyping.
By observing and interviewing forestry workers we found
that their tasks during a working day are very complex. The
need of high knowledge of the forest plus all the decision
that they have to make in every working moment makes
their work hard. The amount of information becomes an
overload and the information that mainly is received as Figure 1. Areas of work overload.
visual information is overwhelming.
To start working with this information is important to
MAPPING THE USERS COGNITIVE WORKLOAD understand the complexity of the things that have to be
When designing objects with a big complexity, the process operated by memory and why some functions are different
of linking the human to machine is demanding. It is than others. The visual information in this type of task is
necessary to start with an analysis in how to interact with both shown by the vehicle screen and in the environment.
three user interfaces, also how cognition and senses are The problem is that only some of the information shown in
involve in these interfaces, and how to use the different the screen is needed to make a decision and the paradox is
types of memory. These analyses will lead us on how the when to show it.
user can access information easier and also showing the
right information at the right time. Phonological strings
Phonology is a subfield of grammar, whereas phonetics is
Combining different types of interfaces improves the about the nature of sounds (or phones). Phonology
process of using elements. An example of this can be stated describes the way sounds function within a given language.
when using a keyboard. You look at the screen and letters T he principles of phonological theory [12] have also been
shown when typing, if you are a skilled writer you don’t applied to the analysis of signed languages, with gestures
look down on the keys. But how do you know where on the and their relationships as the object of study.
keyboard you are located with your fingers? Most of the
keyboards today are equipped with a small rise in the A full sentence has several words that make a complete
surface on the top of the “F” and the “J”, that rise gives you idea. The way the brain identifies the idea is by building a
tactile feedback about where you are located on the network of key words (shortcuts) that together have a
keyboard with your fingers. The rise also works as a tactile meaning. These sounds are called Phonological strings and
memory reminder and you can memorize where to put your are the key sounds that let our brain understand without
fingers if you want to write a certain word. This little trick further explanation.
makes the process of writing with the keyboard much easier Imagine the sound of a Harley Davidson motorcycle. The
and efficient. sound is so specific that the brain of those who know about
In the field of specialized vehicles, users need to develop these motorcycles will recognize the brand without using
different skills to be able to accomplish their job. The first any further sounds related to Harley Davidson.
approach to understand the interaction with different user This metaphor of Phonological strings is the one that
interfaces is by mapping the areas where cognitive load inspired the design solution for handling specialized
increases, finding problems and understanding the vehicles. Designing associations in the different interfaces
complexity of one small sequence of work. allowing the users to do a specific task, without visual
Figure 1 reveals the three areas of work overload, by instructions.
showing visually the fields of interaction in one instant of a
sequence of the users work. The red dots represent the areas
that the users operates when taking one decision, the gray

51
To design these user interfaces we analyzed networks of
complex associations and the different sensory memories
[6] are left do the rest.

Figure 5. Example of working functions

By structuring the information in different areas, related to

the operation of specialized vehicles, it is possible to find
the functions faster. The results are automatic movements
by the operator with the user interface and reduce workload.
Figure 3 shows a group of functions organized on a timeline
basis.

Figure 2. Example of a phonological string leading from

the word “natural phenomenon” to interaction [13]

Figure 2 shows how a word like “natural phenomenon” can

trigger stored information and gather all the associations
connected to it. This example enables us to understand how
the sensory memory works. By saving just the key links the
memory will build the complete thought or picture.
The metaphor was used in this project to design a group of
interfaces, based on links and associations, which help the
brain to build a better image. Through small key icons, Figure 3. Identifying the timeline and the activity of
the different tasks to organize functions.
sounds or tactile elements the brain will link all these pieces
together and generate complex ideas faster. [11] Using the different memories of the brain can make the
work tasks less demanding. The brain is like a vehicle
Integrating three user interfaces where all the information is located. If we only use the front
The solution to the problem encountered in the case of door, it is impossible to access all the information in it, so it
“handling information in specialized vehicles”, was to is necessary to use other doors, i.e. the different memory
speed up the learning process by reducing the cognitive senses.
workload in three ways. By connecting all the elements of a user interface, visual,
By generating an identity and using the same code for tactile and auditive memory, it is possible to reduce the
different user interfaces, associations, it is possible to amount of information needed in a display. If visually this
connect each different icon, sound or tactile interface with information is visually translated into icons, that represent a
the same idea, and allow this graphical image to travel to group of activities, a user can identify a full sequence with
other interfaces with the purpose of retaining the same different functions. Therefore it is not necessary to create
identity. different sets of icons for every single function.
Figure 5 shows how tactile memory and association make it Figure 4 shows each task represented and colored in the
possible to generate a new method of scrolling the different diagram. This way, the development of an identity
groups of functions and structure. generating the association can be expressed visually.

52
 Taking in consideration the different aspects of human
behavior and cultural background can prevent us from
falling in the mistake where humans can no longer
understand objects that surround us. Imagine the old man
that lived through the Second World War, the introduction
of computers and the information technology. How does he
see the world? Different eyes, different knowledge,
different needs. But therefore it doesn’t mean that he is less
capable and doesn’t understand new products and
computers. It’s all about what you are exposed for: cultural,
historical, language, knowledge etc.
ACKNOWLEDGMENTS
We would like to thank the people and organizations that
helped us during the project: Komatsu Forest, Bengt-Arne
Wallden, SLU, Iwan Wästerlund, Carin Larsson, Niklas
Anderson , Mike Stott, Dennis Holmgren, Mikael Sundling,
Figure 4. Connection between graphical, tactile, audio and Fredrik Sundling
user interfaces. REFERENCES
1. 3D Graphical User Interfaces, Farid BenHajji, Erik
The three user interfaces share iconic associations, but to Dybner, Department of Computer and Systems
move from one set of functions to another the user has to Sciences, Stockholm University and The Royal
move the joystick either right or left. In the user’s tactile Institute of Technology, July 1999
memory this movement means the same as it is shown in 2. A Graphical User Interface Framework for Digital
the graphical user interface, what you do is the same as Television, César Pablo and Vuorimaa Petri,
what you see, or hear. Each task will have different Telecommunications Software and Multimedia
representations depending on the graphical, tactile or Laboratory, Helsinki University of Technology, P.O.
auditive user interface but in the brain they will mean the Box 5400 FI-02015 HUT, FINLAND, Tel: +358 9
same. 4515128, Fax: +358 9 4515351
CONCLUSION 3. A. Dix and R. Mancini (1997). Specifying history and
Reducing the amount of information improves imagining backtracking mechanisms. In Formal Methods in
concrete associations and decreasing interference with Human-Computer Interaction, Eds. P. Palanque, Post-
perception in each type of user interface. (Hard to see and web cognition Dix, Howes, Payne 11 and F. Paterno.
imagine at the same time). London, Springer-Verlag. pp. 1-24.
Using metaphors like the phonological strings, helps http://www.comp.lancs.ac.uk/computing/users/dixa/pa
designers understand complex associations and develop pers/histchap97/
solutions that involved massive amount of information. 4. A. Dix, J. Wilkinson and D. Ramduny (1998).
The design process needs to evolve. It is important to think Redefining Organizational, Memory – artefacts, and
about using different senses when developing products, the distribution and coordination of work. In
reducing the cognitive workload, and feeling responsible Understanding work and designing artefacts (York,
for the amount of information objects generate. As 21st Sept., 1998).
designers it is important to take this in consideration when http://www.hiraeth.com/alan/papers/artefacts98/
designing an object. Be more aware of the future and not 5. A. Sloman. Beyond Shallow Models of Emotion.
forgetting the past because that is our present now. Cognitive Processing: International Quarterly of
DISCUSSION Cognitive Science, 2(1):177-198, 2001.
What should be done in the next step is testing the different
6. H. A. Simon. Motivational and emotional controls of
user interfaces together in the right environment with the
cognition, 1967. Reprinted in Models of Thought, Yale
real users.
University Press, 29-38, 1979.
The development of better solutions and communications
7. Hints for a graphical user interface for HADES data
between human and machine, provides better feedback
analysis. P.Finocchiaro°, D.Vasiliev, INFN -
from systems.
Laboratori Nazionali del Sud, via S.Sofia 44, 95123
The design process needs to map and take in consideration, Catania, ITALY, phone: (+39)095.542.284, e-mail:
different aspects like cultural background, age, language, finocchiaro@lns.infn.it
knowledge and human behavior in general, to identify the
8. J.A. Simpson. Attachment theory in modern
different icons and languages that an object communicates
evolutionary perspective. In Handbook of Attachment,
through the users interface.

53
 eds. J. Cassidy & P.R. Shaver, pages 115{141. 11. The Elements of User Experience, jjg@jjg.net, 30
Guilford Press, London, 1999. March 2000 Jesse James Garrett http://www.jjg.net/ia/
9. Post-web cognition: evolving knowledge strategies for 12. The Written language Bias in Linguistics , Per Linell,
global information environments Alan Dix1, Andrew Linköping : Dep. of communication studies Univ.,
Howes2, Stephen Payne2 alan@hcibook.com, 1982 ; http://eserver.org/langs/linell/
howesa@cardiff.ac.uk, paynes@cardiff.ac.uk
13. The Visual Thesaurus software property of the
http://www.hcibook.com/alan/topics/web/
company Thinkmap, Inc. New York, NY, phone: 212-
10. S.K. Card (2002) Cognitive science as the engine of
285-8600, http://www.visualthesaurus.com/
innovation: beyond human-computer interaction. In W.
Gray and C. Schunn (Eds.) Proceedings of the 24th
Annual Conference of the Cognitive Science Society.
August 7-10, George Mason University, Fairfax, VA.

The columns on the last page should be of approximately equal length.

54
Umeå Institute of Design
Designhögskolan i Umeå