KEMBAR78
Cognitive Offloading PDF | PDF | Metacognition | Memory
0% found this document useful (0 votes)
184 views13 pages

Cognitive Offloading PDF

Uploaded by

Uzver
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
184 views13 pages

Cognitive Offloading PDF

Uploaded by

Uzver
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Review

Cognitive Offloading
Evan F. Risko1,* and Sam J. Gilbert2,*
If you have ever tilted your head to perceive a rotated image, or programmed a
Trends
smartphone to remind you of an upcoming appointment, you have engaged in
Physical action is sometimes used to
cognitive offloading: the use of physical action to alter the information process- reduce the cognitive demands of a task.
ing requirements of a task so as to reduce cognitive demand. Despite the This is known as cognitive offloading.

ubiquity of this type of behavior, it has only recently become the target of Recent studies have begun to investi-
systematic investigation in and of itself. We review research from several gate the processes that trigger cogni-
domains that focuses on two main questions: (i) what mechanisms trigger tive offloading, and the cognitive
consequences of this behavior.
cognitive offloading, and (ii) what are the cognitive consequences of this behav-
ior? We offer a novel metacognitive framework that integrates results from Propensity to offload cognition is influ-
enced by the internal cognitive demands
diverse domains and suggests avenues for future research.
that would otherwise be necessary.

Offloading Cognition It is also influenced by metacognitive


evaluations of our mental abilities.
A moment's reflection on our day-to-day cognitive lives reveals the intimate relation between
human cognition and manipulation of the body and objects in the physical environment. We tilt These metacognitive evaluations are
our heads while trying to perceive ambiguous images, we gesture while imagining spatial potentially erroneous, which may lead
transformations, and we rely on smartphones and search engines to store and retrieve infor- to suboptimal offloading behavior.
mation. In other words, we often think using our bodies and the external world. This ability to
flexibly deploy ad hoc mixtures of internal and external processes in pursuit of our cognitive goals
likely represents a defining feature of what it means to be a successful cognitive agent in a
complex environment [1–4]. One crucial function that these mind/body/world interactions afford
is cognitive offloading (see Glossary)–the use of physical action to alter the information
processing requirements of a task so as to reduce cognitive demand (see also computational
offloading [5]; epistemic actions [6]). Our unaided mental abilities have well-known limits (e.g., we
can only accurately perceive a relatively small region of the visual field [7] and can only hold a
limited amount of information active in memory [8]). Offloading cognition helps us to overcome
such capacity limitations, minimize computational effort, and achieve cognitive feats that would
not otherwise be possible. Consistent with this notion, cognitive offloading has been demon-
strated to improve performance across several domains (e.g., perception [9], memory [10],
arithmetic [11], counting [12], and spatial reasoning [13]).

The term cognitive offloading has long existed in the conceptual repertoire of cognitive scientists,
and the phenomenon it refers to is ancient (e.g., finger-counting and abacuses in numerical
cognition, systems of knots or quipus for memory [14]). However, cognitive offloading has rarely
been the target of systematic experimental investigation in and of itself. This has now begun to
change. This change has been precipitated by an increasing interest among cognitive scientists
in ‘wider’ conceptions of cognition (e.g., embodied, embedded, extended, and distributed
approaches [2,3,15–20]). In addition, increased interest in cognitive offloading is emerging at a 1
Department of Psychology, University
time when the opportunity to offload cognition onto technological prostheses has reached a kind of Waterloo, Waterloo, Canada
2
Institute of Cognitive Neuroscience,
of fever pitch–the potential consequences of which (both bad and good) have not gone University College London, London,
unnoticed by the general public (‘Is Google making us stupid?’ [21]). Thus research on cognitive UK
offloading offers both a deeper understanding of the physically distributed nature of human
cognition and translational insights into its potential use (and abuse) in our day-to-day lives. We
*Correspondence:
review here recent research investigating cognitive offloading across three different domains, efrisko@uwaterloo.ca (E.F. Risko) and
focusing on two fundamental issues: (i) what factors influence the likelihood of individuals sam.gilbert@ucl.ac.uk (S.J. Gilbert).

676 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 http://dx.doi.org/10.1016/j.tics.2016.07.002
© 2016 Elsevier Ltd. All rights reserved.
offloading cognition versus relying on internal processes alone, and (ii) what are the cognitive Glossary
consequences of this behavior? Cognitive offloading: the use of
physical action to alter the
Thinking with the Body information processing requirements
of a task so as to reduce cognitive
Cognitive offloading can be roughly subdivided into actions that offload cognitive demands onto- demand.
the-body and into-the-world. We turn first to the former. Recent research in cognitive science has External normalization: the use of
focused on how we actively use our bodies in the ‘here-and-now’ to reduce cognitive demand. physical action (e.g., head tilt) to align
For example, we use our eyes to index locations in space [22], we use our fingers, point, or nod a stimulus with a representation
stored in memory.
our head to mark positions in sequential tasks [12,23], we move our hands to externalize thoughts Feeling-of-knowing: predictions
[11,24,25] and to simulate spatial transformations [13], and we move or shift our body to simplify made by an individual about whether
perceptual computations [9]. In each of these cases an action is spontaneously performed in the they will be able to retrieve specific
information.
context of an ongoing cognitive act so as to generate some form of cognitive savings.
Intention offloading: creation of a
cue in the external environment to
A straightforward example of this type of cognitive offloading is external normalization. For trigger a delayed intention.
instance, when individuals encounter a rotated stimulus (e.g., a tilted book) they often physically tilt Internal normalization: use of an
internal transformation (i.e., mental
their head to normalize its orientation. This behavior is an example of external normalization and
rotation) to align an internal
can be considered as a means to offload internal normalization, which is an internal transfor- representation of a stimulus with a
mation (in this case mental rotation) that aligns a representation of a stimulus with a representation representation stored in memory.
stored in memory [26,27,82]. Indeed, external normalization can reduce the costs of stimulus Metacognition: higher-order
thinking, or ‘thinking about thinking’,
rotation [9]. One of the major tasks in understanding cognitive offloading is to determine the factors
to enable evaluation and control of
that influence whether some external means is integrated into the performance of a given cognitive one's mental processes.
act or not. In the context of external normalization, one of the crucial factors is internal demand. Stroop task: a reaction-time task
Specifically, individuals are more likely to spontaneously physically rotate as the display becomes involving conflict between two
stimulus dimensions (e.g., the color
more disoriented or as the number of items in the display increases [9]. Crucially, both of these and meaning of word stimuli)
manipulations also increase stimulus-rotation costs (i.e., internal demand; a general description of Transactive memory system: a
this methodology is provided in Box 1 and Figure 1A). Thus, as the internal demands associated memory system composed of a
with stimulus rotation increase, the likelihood of spontaneous external normalization also group that collectively encodes,
stores, and retrieves knowledge.
increases. This general pattern has now been observed across several domains (e.g., external
normalization [9], prospective memory [10], short-term memory [28], co-speech gesture [29], and
co-thought gesture [13]; representative examples are given in Figure 2).

While the relation between internal demand and cognitive offloading is robust, they are never-
theless dissociable. This was revealed in an investigation of external normalization using arrays of
words wherein both the words and the frame (i.e., the overall structure of a multi-element array)
were rotated, versus arrays where the words were rotated but presented within an upright frame
[30] (Figure 1A). These two conditions yield similar rotation costs and similar responses on a
physiological measure of demand [30,31]. Nevertheless, spontaneous rates of external normali-
zation are much higher when both the words and the frame are rotated compared to when only
the words are rotated. This dissociation is argued to arise because individuals rely on an
erroneous metacognitive evaluation of demand. This evaluation may be led astray by intuitive
beliefs regarding the effects of stimulus rotation, or a history of external normalization with
displays featuring word and frame rotation. Consistent with this account, individuals incorrectly
report that rotated word and frame displays are more time-consuming and error-prone, and
judge these displays to be more effortful to read than displays with only the words rotated [30].

Putting Cognition Into-The-World


Like offloading cognition onto-our-body, offloading cognition into-the-world is a ubiquitous part
of our everyday cognitive lives [4,14,32,33]. A key way in which we offload cognitive processes
into-the-world is by using it as a repository of representational information, thus eliminating the
need for an internal representation. For example, individuals might write down [28], type into a
computer [34,35], sketch [36], or in some other manner alter the environment so as to record
information that needs to be remembered [14,37]. We discuss examples of this below.

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 677


Box 1. Methods: The Choice/No Choice Paradigm
Research on cognitive offloading has relied heavily on variants of the choice/no choice paradigm [84]. The application of
the paradigm is straightforward: in some conditions participants are obliged (i.e., they have no choice) to employ a
particular strategy and in others they are free (i.e., they have the choice) to select among a set of available strategies. Each
condition and the comparison between conditions provides answers to theoretically interesting questions. In addition,
these conditions are typically paired with one or more other manipulations that influence some variable of interest (e.g.,
memory load). Below we provide an illustrative example using offloading memory onto an external medium (e.g., a piece
of paper or a computer).

No Choice–Internal

Individuals are tasked with remembering a given piece of information without being able to store it externally. This
condition provides a measure of performance when the individual has to rely solely on their internal memory.

No Choice–External

Individuals are tasked with remembering a given piece of information and must store it externally. This condition provides
a measure of performance when the individual uses external memory. It is important to note that unlike the ‘no choice–
internal’ condition this condition cannot ensure that the information is not also stored internally. The comparison of the
two ‘no choice’ conditions provides a measure of the relative effectiveness of storing information internally and externally.
This comparison is often made as a function of some other variable (e.g., the amount of to-be-remembered information).

Choice

Participants are allowed to freely choose between storing information internally or externally. This condition provides a
measure of the spontaneous offloading of memory demands onto the external medium. Again, how the spontaneous
offloading of memory demands changes as a function of some other variable (e.g., the amount of to-be-remembered
information) is typically of interest. This condition also provides a measure of performance when the individual uses their
‘preferred’ strategy.

Challenges

The choice/no choice paradigm is not without challenges. As noted above, attempting to oblige individuals to adopt a
strategy might not be effective in some circumstances. In addition, forcing individuals to use a particular strategy could
introduce demands associated with being required to inhibit the use of another possibly preferred strategy. For example,
restricting individuals from gesturing could impose its own load associated with inhibiting naturally occurring gestures
[11].

Offloading Memory–Prospective Memory


We rely on memory not only to recall information from the past but also to execute intended
behaviors in the future. Our ability to remember delayed intentions is termed ‘prospective
memory’ [38,39]. Everyone is familiar with its fallibility: failures of prospective memory probably
comprise a majority of self-reported everyday memory problems [40]. What makes remembering
delayed intentions particularly difficult is that, in many cases, our intentions are not effectively
triggered by perceptual cues in our environment, and therefore action must be self-initiated. It is
therefore unsurprising that people have long supported prospective memory by using external
tools to supply perceptual cues that can trigger intended actions. Examples include tying knots in
handkerchiefs, placing reminders in the environment (e.g., post-it notes or task-relevant objects),
or–in recent times–using smartphones or wearable devices that can provide time-, location-, or
person-based reminders [41,42]. This form of cognitive offloading–acting on the environment to
create external triggers for delayed intentions–has been referred to as intention offloading
[10,43]. Laboratory studies of prospective memory have generally considered our tendency to
outsource intentions to external tools as a source of noise that obscures ‘real’ prospective
memory processes, and prevented individuals from setting external reminders (e.g., [44]).
However, intention offloading is likely central to our ability to remember intentions in the real
world, and is hence an important topic for investigation. This process was investigated empirically
in a recent series of behavioral [10,43] and neuroimaging [45] studies illustrated in Figure 1B.

678 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9


(A) External Normalizaon

RD
WO

RD
RD

RD

RD
WORD WORD WORD

WO
WO

WO

WO

RD
RD

WO
WO
RD

RD

RD

RD
WORD WORD WORD

WO

WO

WO

WO

RD
RD

WO
RD

RD

RD

WO
WORD WORD WORD

RD
WO

WO

WO
Ro
tat

WO
ed

RD
Upright words Rotated words wo
rds

WO
and
fra
me

Internal normalizaon External normalizaon

(B) Intenon Offloading

A: Instrucons
B: Inial smuli
C: Intenon offloading
(oponal)
3
Please drag the numbers in order to the
4
boom of the box (1, 2, 3, 4, etc.) 3
5
4
BUT: 3
6 5
8
Please drag 3 to the TOP instead. 7
9 6
8
2 1 7
9
10 2 1

10

D: Ongoing response
3 E: Ongoing response
3 F: Target response
4
3
5
4
6 5
8 4
7
9 6 5
8
2 1 7
6
9 8
10 2 7
9
10

10

Figure 1. Paradigms for Investigating Cognitive Offloading. (A) In the external normalization paradigm [9] participants
read arrays of words that are presented in upright or rotated orientations. When faced with rotated words, participants can
align them using internal cognitive processes (‘internal normalization’) or physical action (‘external normalization’). (B) In the
intention offloading paradigm [10] participants use a mouse or touchscreen to drag numbered circles in sequence to the
bottom of the screen. They are also instructed at the beginning of the trial that one or more of these circles should be
dragged to an alternative location. They can either remember these intentions internally or offload them by dragging target
circles toward their intended location at the beginning of the trial. In some ways this is analogous to everyday offloading
behavior such as leaving an item by the front door so that we will remember it when leaving the house. For a demonstration
of the task, please visit ‘http://samgilbert.net/offloadDemo.html’. Illustrations modified, with permission, from [9] and [45].

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 679


(A) 1.70 34
1.60 32
1.50 30
Free rotaon Restricted moon
1.40 28
condions condions
1.30 26
1.20 24
1.10 22
1.00 20

ms/Degree
%/Degree

0.90 18
0.80 16
0.70 14
0.60 12
0.50 10
0.40 8
0.30 6
0.20 4
0.10 2
0.00 0
–0.10 –2
1 Leer 5 Leers 15 Leers Paragraph 1 Leer 5 Leers 15 Leers Paragraph
Experiment 1 Experiment 3 Experiment 2 Experiment 3

(B) Accuracy without intenon offloading Intenons offloaded when possible


80
95

60
90

85 40
%

80
20
75

70 0
No interrupon Interrupon No interrupon Interrupon

1 intenon 3 intenons

(C) Accuracy Choice behavior


100 100

90 90

80 80

70 70

60 60
% Correct

% Write

50 50

40 40

30 30
Choice | Wrote
20 20
Choice all trials
10 No choice 10

0 0
2 4 6 8 10 2 4 6 8 10
Set size

Figure 2. Relation between Internal Demand and Cognitive Offloading. There exists a consistent relation between
the amount of internal demand, as indexed in a condition where offloading is restricted, and the amount of spontaneous
offloading behavior observed in a condition where the behavior is not restricted. This has been demonstrated across several
different domains. With respect to external normalization (A), as the internal costs of stimulus rotation increase when
individuals are forced to remain upright (i.e., see rotation costs in ms/degree in ‘Restricted motion conditions’; larger values
represent greater costs), the likelihood that an individual spontaneously physical rotates increases (i.e., see ‘Free rotation
conditions’; larger values represent a higher frequency of offloading [9]). In intention offloading (B) and short-term memory
(C), as the unaided memory performance decreases (see ‘Accuracy without intention offloading’ in B and ‘Accuracy: no
choice’ in C), the likelihood that an individual spontaneously offloads the memory demands into the environment (i.e., setting
reminders; writing the to-be-remembered items down) increases (see ‘Intentions offloaded when possible’ in B and ‘Choice
behavior’ in C; in both cases higher values represent a higher frequency of offloading [10,28]). Graphs modified, with
permission, from [9], [28], and [43].

680 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9


Box 2. Perspectives on Cognitive Impartiality
In discussions about cognitive offloading a central question arises with respect to whether the cognitive system has an
inherent bias away from specific types of effort. For example, in selecting between storing information in short-term
memory (i.e., in-the-head) or writing that information down (i.e., in-the-world [28]) individuals are selecting the type of
effort that will be required to carry out the task–more internal or cognitive effort in the former case, and external or
perceptual/motor effort in the latter case. Two views have dominated discussions of this issue ([2] for further discussion).
On the cognitive impartiality view the cognitive system has no bias or is indifferent to the type of effort required. For
example, according the soft constraints hypothesis [85,86], it is not the type of effort but rather the amount of time
required that determines the preferred solution (i.e., the solution with the shorter time being the preferred one). An
alternative view, which might be called the ‘cognitive miser’ view, is that individuals have an inherent bias against
expending cognitive effort. There has been much recent work on the tendency of individuals to avoid this type of effort
[87,88]. One influential theoretical position that embodies this view is the ‘minimal memory’ view [22], according to which
the cognitive system is biased toward minimizing demands on memory (even in the face of potentially greater perceptual-
motor costs). Between these theoretical signposts likely lie several interesting alternatives; for example, individuals may
have idiosyncratic biases in one direction or the other, or variable task-dependent biases. Future work aimed at
adjudicating between these and related views will provide deeper understanding of the how the cognitive system
distributes resources across brain, body, and world.

Similarly to tilting one's head to read rotated text, intention offloading is influenced by
the internal demands that would otherwise be necessary (Figure 2). Individuals are more
likely to offload intentions when their memory load increases or when they encounter
interruptions; both these factors impair performance when offloading is prevented [10].
However, and again analogously to external normalization, intention offloading is not only
driven by objective need but also by a potentially erroneous metacognitive evaluation of
demand. This was demonstrated in a study where individuals remembered delayed
intentions both with and without the ability to set reminders, and also provided predictions
about their performance. Individuals with lower confidence in their memory abilities were
more likely to spontaneously set reminders, even after controlling for any influence of
objective ability (which also predicted intention offloading [43]). Interestingly, this relation
with metacognitive confidence is domain-general. When individuals performed a separate
perceptual judgment task where accuracy was held constant with a staircase procedure,
individuals with lower confidence in their perceptual judgments set more reminders in the
intention offloading task [43]. Thus, intention offloading is related not only to individual
differences in objective ability but also to domain-specific and domain-general metacognitive
confidence.

Once an individual has opted to offload, what are the consequences for information
processing? In the context of intention offloading, placing information into the external
environment brings several potential benefits. One of the most salient is that offloaded
representations may be more durable and less prone to distortion than those stored
internally, leading to an increased likelihood of intention fulfillment [10,43]. However, it is
important to note that individuals also set reminders in conditions where doing so led to no
objective increase in accuracy [10,43]. This also occurs in the context of external normali-
zation [9]. This tendency to engage in offloading despite it not benefiting performance may
result from (i) an undetected performance benefit (ii) a bias against cognitive effort (Box 2),
and/or (iii) an erroneous metacognitive belief that the offloading will in fact benefit perfor-
mance. Support for the latter interpretation comes from recent research examining offloading
in a short-term memory task [28]. Participants were allowed to offload to-be-remembered
materials (i.e., by writing them down) and did so about 40% of the time when they were
required to remember only two items, a memory load at which performance was already at
ceiling without offloading. Crucially, individuals erroneously judged that offloading would
improve their performance in this latter condition. Thus, the putatively superfluous offloading
(observed across several domains) underlines again the importance of metacognitive beliefs
in cognitive offloading.

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 681


Offloading Memory–Transactive memory
While the research reviewed thus far has focused little on the cognitive consequences of
offloading, recent research on transactive memory has made this issue its primary focus. In
transactive memory systems, knowledge is distributed across two or more individuals such
that the system as a whole knows more than any one individual [46–48]. Recent research has
extended this notion of socially distributed memory to human-technology transactive memory
systems [34,49–52]. Our ability to reliably store and (almost) instantaneously retrieve information
has changed drastically with the advent of the computer and the internet. Consequently we can
now offload much of what in the past would have been stored internally.

To examine the idea that offloading might impair our memory, in one study individuals were
presented with a series of trivia statements to remember and were asked to type them into a
computer. In addition, half the individuals expected that the information would be saved and half
expected it to be erased [34]. Recall tests demonstrated that those in the latter condition had
better memory than the former. The authors argued that memory-encoding demands were
offloaded onto the external store, leading to memory impairments when it was not available ([34];
see Box 3 for additional costs of cognitive offloading). Interestingly, these offloading-based
memory impairments can be accompanied by enhanced memory for other information. For
example, when individuals saved an initial list of words it enhanced memory for a second list [35].
The authors argued that saving reduced the likelihood that the first list of words interfered with
memory for the second (i.e., reduced proactive interference; see also [53]).

Offloading memory demands in a transactive system is not a ‘free pass’ in terms of mnemonic
requirements. Instead, a defining attribute of a transactive memory system is a shift from
remembering ‘what’ to remembering ‘where’. For example, when you offload information about
a meeting to a file on your computer, you no longer need to remember the content of the file, but
you do need to remember where to find it. Consistent with this idea, saving an external file can
lead to an enhanced ability to recall where to find information, at the expense of remembering
what it actually is ([34]; for an alternative explanation see [53]). Similarly, when faced with a failure
to recall memory content, thoughts about memory location can be primed relatively automati-
cally. This was demonstrated in a study where individuals answered easy or difficult trivia
questions, then completed a variant of the Stroop task [34]. Stroop-like interference from
words relating to internet search engines was increased after individuals answered difficult
compared with easy questions, consistent with those terms being primed in individuals’ minds.

Beyond its influence on memory, being part of a human-technology transactive memory system
can also have subtle effects on metacognition. For example, searching for information online
about one topic can lead individuals to believe that they have more knowledge ‘in-the-head’ and
generate more ‘brain activity’ when answering questions about another topic [49]. In a separate
line of experiments, individuals who had recently used Google to help them complete a quiz
reported higher levels of cognitive self-esteem. They also predicted that they would do better on
a subsequent quiz, even without help from external resources [50,51]. These results suggest
that participating in a human–internet transactive memory system can lead individuals to blur the
distinction between what they know and what the internet ‘knows.’ However, this outcome does
not occur in all circumstances. In another study, participants had to report whether they knew
the answer to a general knowledge question or not. In one condition, if participants responded
that they did not know the answer, they looked it up on the internet. In a second condition, if
participants responded that they did not know the answer, they simply moved on to the next
question. Thus, participants had access to the internet in one condition and no access in the
other. Crucially, when they knew they would subsequently have access to the internet, par-
ticipants were more likely to answer ‘don’t know’ and reported lower feeling-of-knowing to the
trivia questions [52]. Thus, internet access in this context reduced individuals’ willingness to offer

682 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9


Box 3. Beyond Google: The Costs of Offloading
GPS

Many people now travel using global positioning systems. Offloading wayfinding onto such a device has been
demonstrated to impair spatial memory [65–67]. For example, in one study individuals who drove a predetermined
route using a turn-by-turn navigation system outperformed individuals who had no aid. However, individuals in the former
group had poorer memory for scenes from the route and, when asked to drive the route a second time without an aid,
performed more poorly [67].

Cameras

In an examination of the influence of taking a picture on memory, individuals visited several objects and either took a
picture or simply observed the object [89]. Memory for the objects tested a day later revealed impaired memory for the
photographed objects. In a subsequent experiment, taking a picture of only part of the object, rather than the whole
object, to some extent ameliorated this cost [89]. It was argued that the act of taking a photograph led individuals to
offload the memory for the object onto the camera [89]. The impairment observed here is particularly interesting because
individuals did not necessarily expect to have the pictures available during the memory test. Thus, the de-prioritization of
information that is potentially available externally might occur spontaneously [90].

Automation

In many cases the decision to offload is not made by the individual. Instead, offloading is ‘built-in’ to the task environment
by design. This could reflect a desire to increase usability [91,92] or automate tasks entirely [93–95]. With respect to
offloading associated with automation, research has focused on two costs that have been observed across several
safety-critical situations (e.g., aviation, medicine, driving), specifically ‘automation complacency’, the failure to be
sufficiently vigilant with respect to the performance of automated processes; and ‘automation bias’, the tendency to
uncritically rely on the output of an automated decision aid [93]. The long-term reliance on automated processes could
also lead to cognitive ‘skill decay’ where a developed ability deteriorates over time [96–98]. Recent research has
highlighted the fact that the consequences of automation on performance can be tied closely to how individuals allocate
resources freed up by automation. For example, driving a highly automated vehicle can improve situation awareness
relative to manual driving if individuals are motivated to attend to the environment, but can impair it if they decide to devote
‘freed resources’ to driving-unrelated tasks [99].

an answer to a question based on their own knowledge. Taken together, this research under-
scores the fact that opportunities to offload cognition can affect both lower-level cognitive
systems (e.g., memory) and higher-level metacognitive evaluations of those systems (e.g.,
confidence).

Metacognition of the Extended Mind


The reviewed research suggests that theorizing on cognitive offloading may benefit from further
investigation of the metacognitive aspects of both the processes that trigger cognitive offloading
and the consequences of this behavior. We offer a framework to support this effort here
(Figure 3, Key Figure). This framework describes situations in which there are two or more
ways of achieving a goal, one of which involves cognitive offloading and one of which does not. In
these circumstances, offloading represents a type of strategy to achieve some cognitive goal,
and follows a strategy-selection phase [54–58,83]. This strategy-selection phase is influenced
by a metacognitive evaluation of the available options (arrow A in Figure 3). In particular, it is
informed by metacognitive beliefs (relating to the person, task, and strategy) and experiences (e.
g., effort [30,32,59,60,83]) that are associated with internal and more ‘extended’ strategies (i.e.,
those integrating an external body- or physical environment-based resource). For example,
when faced with a need to remember a given piece of information, our knowledge regarding our
previous success with internal (e.g., metacognitive confidence) and external storage [43,61,62],
our beliefs about the reliability of a particular external store [35], and/or a feeling of fluency could
all contribute to whether an individual stores that information internally or offloads the memory
demands into-the-world. This framework places at center stage a need for a deeper under-
standing of the metacognitions associated with cognitive offloading, and generates several
interesting avenues for future research (see Outstanding Questions).

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 683


Key Figure
A Metacognitive Model of Cognitive Offloading

Lower-level cognive processes (e.g., memory) Figure 3. We propose that selecting


(C) between offloading and relying on internal
processes is influenced by metacognitive
Metacognive beliefs and experiences (e.g., effort) evaluations of our (internal) mental capa-
cities and the capacities of our extended
(B) mental systems encompassing body and
world (arrow A). An example of this would
be evaluating our unaided spatial memory
and a GPS system when deciding how to
vs navigate to a friend's house. In addition,
engaging in either internal or extended
strategies can influence subsequent
metacognitive evaluations (arrow B). For
instance, after successful use of a GPS
Metacognive evaluaon of system we may come to believe that it is a
internal vs extended strategies more reliable guide than our unaided
memory. Offloading can also directly
(A) impact on our lower-level cognitive pro-
Strategy selecon cesses (arrow C). An example of this
would be a reduction in our internal spatial
memory for a location after reliance on
Engaging in internal vs extended strategies GPS navigation.

It is important to note that the strategy-selection phase postulated above does not necessarily
imply that individuals are aware of making a choice ([63,64] for discussion of this issue). Clearly,
there is a range of situations that putatively involve cognitive offloading, some of which involve
conscious deliberation and others of which do not. For example, gesture, which is often
associated with cognitive offloading, can occur without individuals necessarily being aware
of it. By contrast, choosing between navigating based on stored knowledge versus plugging a
set of coordinates into a GPS device is likely more strongly associated with a phenomenology of
deliberation and choice. Thus, an important question within the proposed framework will be to
examine the extent to which different forms of cognitive offloading involve conscious deliberation
or not, and how these cases are similar or distinct.

Our framework also attempts to capture the downstream effects of cognitive offloading on how
we think. As reviewed above, recent work has demonstrated that the experience of offloading
cognition [49–51] and the opportunity to do so [52] can in and of itself alter our thinking about our
internal capacities (i.e., our metacognitions; arrow B in Figure 3). For example, offloading
information retrieval onto the internet can inflate our estimates of our own knowledge [49–
51]. In addition, this work has demonstrated that cognitive offloading can have both costs and
benefits with respect to basic cognitive processes (arrow C in Figure 3). For example, offloading
to-be-remembered information can both aid and impair retrieval from internal memory stores
[34,35]. It should also be noted that, beyond reducing cognitive demand, offloading could also
qualitatively change the processes involved in thinking, communicating, and learning, potentially
with both positive and negative consequences [13,14].

The metacognitive framework offered here also highlights potential interactions between off-
loading and the mechanisms that trigger this behavior. For example, deciding whether to rely on

684 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9


a GPS device for the location of a friend's house versus our internal memory will be informed by Outstanding Questions
beliefs in the relative accuracy of each method (a computation that will likely favor the former While much work has focused on cog-
strategy; arrow A). The tendency to offload wayfinding to the more accurate GPS will likely nitive offloading's influence on cogni-
tion in the short-term, much less has
reduce both our internal spatial memory for that location (arrow C) [65–67] and our metacog- focused on potential long-term effects.
nitive confidence in it (arrow B), which will in turn increase the likelihood that we choose to rely on How does offloading over an extended
the external artefact in the future (arrow A). Thus the model predicts a type of self-reinforcing period of time impact on our unaided
mental abilities (does Google make us
pattern that will produce a drift away from reliance on internal capabilities when situated in an
stupid)? How will it influence our meta-
environment with effective cognitive technologies ([68] for an example of this type of drift in the cognitive evaluation of our abilities?
context of Inuit wayfinding). Understanding the long-term cognitive consequences of this drift
represents an important area of future research. Most work investigating cognitive off-
loading has focused on adults. How do
cognitive offloading strategies emerge
Practical Implications
developmentally, and how do they
Research investigating cognitive offloading has clear practical implications–two of which we change across the lifespan?
highlight here. First, individuals with impaired unaided cognitive ability may particularly benefit
from cognitive offloading. How can those who would benefit the most be encouraged to do so There are large individual differences in
[69]? The metacognitive model of cognitive offloading put forward in this article suggests that individuals’ use of offloading strategies.
To what extent are individual differen-
compensatory offloading strategies are most likely to be adopted in individuals with metacog-
ces in decisions to offload cognition
nitive awareness of their impairment. This implies potential challenges in populations with domain-specific or domain-general?
metacognitive difficulties, for example in cases of acquired brain injury where there can be a
mismatch between an individual's metacognitive evaluation of their abilities–built up over a Cognitive offloading comes in many
lifetime–and the post-injury reality [70,71]. Improving metacognitive insight in cases such as forms. Are these different forms of off-
loading (e.g., writing with pen and
these could lead to more-appropriate compensatory offloading [72] paper versus storing a note in a com-
puter, retrieving information from the
The second general area in which research on cognitive offloading has important practical internet versus a friend) cognitively
implications is education [11,24,73,74]. There has long been interest in the potential utility of equivalent?

educational interventions and aids that allow children to offload some of the cognitive demand
What neural mechanisms are respon-
while learning (e.g., manipulatives [73,74], calculators [75,76]). For example, gesture helps sible for triggering cognitive offloading
children learn by ‘lightening the load’ [11] and, interestingly, this benefit appears to outstrip strategies, and what are the neural
that garnered by offloading demands onto external manipulatives [77]. The latter suggests the consequences?
need to consider whether different forms of offloading might have different educational con-
Given the potential costs and benefits
sequences. Crucially, any benefit of offloading will be contingent on the fact that the demand
of offloading, it may be useful to
being offloaded is unnecessary with respect to the learning goal (see [78] for relevant distinctions encourage individuals to offload more
between necessary/intrinsic/germane load and unnecessary/extraneous load in learning). In or less than they currently do. What
addition, it is important that what is ‘saved’ by offloading is redistributed productively rather than interventions can increase or reduce
individuals’ propensity to engage in
being re-allocated to superfluous activities (e.g., intentional mind-wandering [79]; Box 2 presents
cognitive offloading?
a similar issue in the case of automating driving).
Offloading strategies arguably aug-
Concluding Remarks and Future Directions ment our cognitive fitness, thus raising
Cognitive offloading represents one of the quintessential examples of how we use our body questions about their role in our evolu-
tionary history. For example, what are
and objects in the external world to help us think. As such, understanding this phenomenon
the evolutionary origins of cognitive
provides a window into the distributed nature of human cognition. It is clear from the present offloading?
review that offloading can take many forms, but that common patterns exist across domains.
In particular, the evidence reviewed above shows that internal demand and metacognitive Do other species engage in cognitive
evaluations of demand play a crucial role in offloading. Furthermore, cognitive offloading offloading and, if so, how?
can have downstream effects on our low-level cognitive capacities and our subsequent
Does the cognitive system have an
metacognitions. We have suggested that an important future direction for this research will
inherent bias toward cognitive versus
be to better understand the metacognitive processes involved in cognitive offloading, and we non-cognitive forms of effort, or is the
have offered a framework to guide this effort. Beyond metacognition, there is a clear need to system ‘cognitively impartial’ (Box 2)?
better understand how offloading demands onto various technologies (e.g., computers,
internet, GPS) impact on our organic abilities both in the short- and long-term. The latter
represents a particularly pressing concern both for researchers and society in general as our
lives come to be more cognitively entangled with these technologies. Conducting this

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 685


needed research, however, is not without challenge. For example, investigating cognitive
offloading often requires allowing research participants to move their body and manipulate
and interact with their environment. Methods in cognitive science, however, have traditionally
been designed to restrict this type of natural behavior [80,81]. Thus, understanding cognitive
offloading will require an expansion of the cognitive scientist's methodological toolbox. This
and other challenges notwithstanding, future research investigating cognitive offloading
promises a deeper understanding of one of the defining attributes of human cognition.

Acknowledgments
E.F.R. was supported by funding from the Canada Research Chairs program (056562), the Natural Science and
Engineering Research Council (57109), and an Early Researcher award from the Province of Ontario (058402). S.J.G.
is supported by a Royal Society University Research Fellowship.

References
1. Barrett, L. (2011) Beyond the Brain: How Body and Environ- 24. Goldin-Meadow, S. and Wagner, S.M. (2005) How our hands help
ment Shape Animal and Human Minds, Princeton University us learn. Trends Cogn. Sci. 9, 234–241
Press 25. Alač, M. and Hutchins, E. (2004) I see what you are saying: action
2. Clark, A. (2010) Supersizing the Mind, Oxford University Press as cognition in fMRI brain mapping practice. J. Cogn. Cult. 4, 629–
3. Pfeifer, R. and Bongard, J. (2006) How the Body Shapes the Way 661
We Think: A New View of Intelligence, MIT Press 26. Jolicoeur, P. (1988) Mental rotation and the identification of dis-
4. Donald, M. (1991) Origins of the Modern Mind: Three Stages in the oriented objects. Can J. Psychol. 42, 461–478
Evolution of Culture and Cognition, Harvard University Press 27. Graf, M. (2006) Coordinate transformations in object recognition.
5. Scaife, M. and Rogers, Y. (1996) External cognition: how do Psychol. Bull. 132, 920–945
graphical representations work? Int. J. Hum-Comput. St. 45, 28. Risko, E.F. and Dunn, T.L. (2015) Storing information in-the-world:
185–213 metacognition and cognitive offloading in a short-term memory
6. Kirsh, D. and Maglio, P. (1994) On distinguishing epistemic from task. Conscious Cogn. 36, 61–74
pragmatic action. Cognitive Sci. 18, 513–549 29. Melinger, A. and Kita, S. (2007) Conceptualisation load triggers
7. Simons, D.J. and Levin, D.T. (1997) Change blindness. Trends gesture production. Lang Cognitive Proc. 22, 473–500
Cogn. Sci. 1, 261–267 30. Dunn, T.L. and Risko, E.F. (2015) Toward a metacognitive account
8. Cowan, N. (2010) The magical mystery four: how is working of cognitive offloading. Cognitive Sci. Published online August 26,
memory capacity limited, and why? Curr. Dir. Psychol. Sci. 19, 2015. http://dx.doi.org/10.1111/cogs.12273
51–57 31. Dunn, T.L. et al. (2016) Metacognitive evaluation in the avoidance
9. Risko, E.F. et al. (2014) Rotating with rotated text: a natural of demand. J. Exp. Psychol. Human. Published online April 28,
behavior approach to investigating cognitive offloading. Cognitive 2016. http://dx.doi.org/10.1037/xhp0000236
Sci. 38, 537–564 32. Schönpflug, W. (1986) The trade-off between internal and external
10. Gilbert, S.J. (2015) Strategic use of reminders: influence of both information storage. J. Mem. Lang. 25, 657–675
domain-general and task-specific metacognitive confidence, inde- 33. Intons-Peterson, M.J. and Fournier, J. (1986) External and internal
pendent of objective memory ability. Conscious Cogn. 33, 245– memory aids: when and how often do we use them? J. Exp.
260 Psychol-Gen. 115, 267–280
11. Goldin-Meadow, S. et al. (2001) Explaining math: gesturing light- 34. Sparrow, B. et al. (2011) Google effects on memory: cognitive
ens the load. Psychol. Sci. 12, 516–522 consequences of having information at our fingertips. Science
12. Carlson, R.A. et al. (2007) What do the hands externalize in simple 333, 776–778
arithmetic? J. Exp. Psychol. Learn. 33, 747 35. Storm, B.C. and Stone, S.M. (2015) Saving-enhanced memory:
13. Chu, M. and Kita, S. (2011) The nature of gestures’ beneficial the benefits of saving on the learning and remembering of new
role in spatial problem solving. J. Exp. Psychol. Gen. 140, information. Psychol. Sci. 26, 182–188
102–116 36. Tversky, B. (2011) Visualizing thought. Top. Cogn. Sci. 3, 499–535
14. Nestojko, J.F. et al. (2013) Extending cognition to external agents. 37. Tversky, B. (2015) The cognitive design of tools of thought. Rev.
Psychological Inquiry 24, 321–325 Philos. Psychol. 6, 99–116
15. Glenberg, A.M. (2010) Embodiment as a unifying perspective for 38. Brandimonte, M.A. et al. (2014) Prospective Memory: Theory and
psychology. Wiley Interdiscip. Rev.: Cogn. Sci. 1, 586–596 Applications, Psychology Press
16. Clark, A. and Chalmers, D. (1998) The extended mind. Analysis 58, 39. Einstein, G.O. and McDaniel, M.A. (1990) Normal aging and pro-
7–19 spective memory. J. Exp. Psychol. Learn. 16, 717–726
17. Hutchins, E. (1995) Cognition in the Wild, MIT press 40. Kliegel, M. and Martin, M. (2003) Prospective memory research:
18. Cowley, S.J. and Vallée-Tourangeau, F. (2013) Cognition Beyond why is it relevant? Int. J. Psychol. 38, 193–194
the Brain: Computation, Interactivity and Human Artifice, Springer 41. McDonald, A. et al. (2011) Google calendar: a new memory aid to
19. Kirsh, D. (1996) Adapting the environment instead of oneself. compensate for prospective memory deficits following acquired
Adapt. Behav. 4, 415–452 brain injury. Neurosychol Rehabil. 21, 784–807

20. Kirsh, D. (2010) Thinking with external representations. Ai & Soci- 42. Svoboda, E. and Richards, B. (2009) Compensating for antero-
ety 25, 441–454 grade amnesia: a new training method that capitalizes on emerg-
ing smartphone technologies. J. Int. Neuropsych. Soc. 15, 629–
21. Carr, N. (2008) Is Google making us stupid? Atlantic Monthly 302,
638
56–62 July/August
43. Gilbert, S.J. (2015) Strategic offloading of delayed intentions into
22. Ballard, D.H. et al. (1997) Deictic codes for the embodiment of
the external environment. Q. J. Exp. Psychol. 68, 971–992
cognition. Behav. Brain Sci. 20, 723–742
44. Uttl, B. and Kibreab, M. (2011) Self-report measures of pro-
23. Alibali, M.W. and DiRusso, A.A. (1999) The function of gesture in
spective memory are reliable but not valid. Can J. Psychol. 65,
learning to count: more than keeping track. Cognitive Dev. 14,
57–68
37–56

686 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9


45. Landsiedel, J. and Gilbert, S.J. (2015) Creating external reminders 72. Cicerone, K.D. et al. (2000) Evidence-based cognitive rehabilita-
for delayed intentions: dissociable influence on ‘task-positive’ and tion: recommendations for clinical practice. Arch. Phys. Med.
‘task-negative’ brain networks. NeuroImage 104, 231–240 Rehab. 81, 1596–1615
46. Wegner, D.M. (1995) A computer network model of human trans- 73. Martin, T. and Schwartz, D.L. (2005) Physically distributed learn-
active memory. Soc. Cognition 13, 319–339 ing: adapting and reinterpreting physical environments in the
47. Harris, C.B. et al. (2014) Couples as socially distributed cognitive development of fraction concepts. Cognitive Sci. 29, 587–625
systems: remembering in everyday social and material contexts. 74. Pouw, W.T. et al. (2014) An embedded and embodied cognition
Memory Studies 7, 285–297 review of instructional manipulatives. Educ. Psychol. Rev. 26, 51–
48. Peltokorpi, V. (2008) Transactive memory systems. Rev. Gen. 72
Psychol. 12, 378–394 75. Hembree, R. and Dessart, D.J. (1986) Effects of hand-held cal-
49. Fisher, M. et al. (2015) Searching for explanations: how the internet culators in precollege mathematics education: a meta-analysis. J.
inflates estimates of internal knowledge. J. Exp. Psychol. Gen. Res. Math Educ. 83–99
144, 674 76. Ellington, A.J. (2003) A meta-analysis of the effects of calculators
50. Ward, A.F. (2013) Supernormal: how the internet is changing our on students’ achievement and attitude levels in precollege math-
memories and our minds. Psychol. Inq. 24, 341–348 ematics classes. J. Res. Math. Educ. 433–463

51. Wegner, D.M. and Ward, A.F. (2013) How Google is changing your 77. Novack, M.A. et al. (2014) From action to abstraction using the
brain. Sci. Am. 309, 58–61 hands to learn math. Psychol. Sci. 25, 903–910

52. Ferguson, A.M. et al. (2015) Answers at your fingertips: access to 78. Paas, F. et al. (2003) Cognitive load theory and instructional
the internet influences willingness to answer questions. Conscious design: recent developments. Edu. Psychol. 38, 1–4
Cogn. 37, 91–102 79. Seli, P. et al. (2016) Mind-wandering with and without intention.
53. Eskritt, M. and Ma, S. (2014) Intentional forgetting: note-taking as Trends Cogn. Sci. 20, 605–617
a naturalistic example. Mem. Cognition 42, 237–246 80. Kingstone, A. et al. (2008) Cognitive ethology: a new approach for
54. Payne, J.W. et al. (1988) Adaptive strategy selection in decision studying human cognition. Brit. J. Psychol. 99, 317–340
making. J. Exp. Psychol. Learn. 14, 534–552 81. Risko, E.F. et al. (2016) Breaking the fourth wall of cognitive
55. Reder, L.M. (1987) Strategy selection in question answering. science real-world social attention and the dual function of gaze.
Cognitive Psychol. 19, 90–138 Curr. Dir. Psychol. Sci. 25, 70–74
56. Walsh, M.M. and Anderson, J.R. (2009) The strategic nature of 82. Shepard, R.N. and Metzler, J. (1971) Mental rotation of three-
changing your mind. Cognitive Psychol. 58, 416–440 dimensional objects. Science 171, 701–703

57. Marewski, J.N. and Schooler, L.J. (2011) Cognitive niches: an 83. Nelson, T.O. and Narens, L. (1990) Metamemory: a theoretical
ecological model of strategy selection. Psychol. Rev. 118, 393– framework and new findings. Psychol. Learn Motiv. 26, 125–141
437 84. Siegler, R.S. and Lamaire, P. (1997) Older and younger adults’
58. Karpicke, J.D. (2009) Metacognitive control and strategy selec- strategy choices in multiplication: testing predictions of ASCM
tion: deciding to practice retrieval during learning. J. Exp. Psychol- using the choice/no-choice method. J. Exp. Psychol. Gen. 126,
Gen. 138, 469–486 71–92

59. Flavell, J.H. (1979) Metacognition and cognitive monitoring: a new 85. Gray, W.D. and Boehm-Davis, D.A. (2000) Milliseconds matter:
area of cognitive-developmental inquiry. Am. Psychol. 34, 906– an introduction to microstrategies and to their use in describing
911 and predicting interactive behavior. J. Exp. Psychol. Appl. 6,
322–335
60. Arango-Muñoz, S. (2013) Scaffolded memory and metacognitive
feelings. Rev. Philos. Psychol. 4, 135–152 86. Gray, W.D. et al. (2006) The soft constraints hypothesis: a rational
analysis approach to resource allocation for interactive behavior.
61. Lovett, M.C. and Anderson, J.R. (1996) History of success and
Psychol. Rev. 113, 461–482
current context in problem solving: combined influences on oper-
ator selection. Cognitive Psychol. 31, 168–217 87. Kool, W. et al. (2010) Decision making and the avoidance of
cognitive demand. J. Exp. Psychol. Gen. 139, 665–682
62. Patrick, J. et al. (2015) The influence of training and experience on
memory strategy. Mem. Cognition 43, 775–787 88. Kurzban, R. et al. (2013) An opportunity cost model of subjective
effort and task performance. Behav. Brain Sci. 36, 661–679
63. Reder, L.M. and Schunn, C.D. (1996) Metacognition does not
imply awareness: strategy choice is governed by implicit learning 89. Henkel, L.A. (2014) Point-and-shoot memories the influence of
and memory. In Implicit Memory and Metacognition (Reder, L.M., taking photos on memory for a museum tour. Psychol. Sci. 25,
ed.), pp. 45–78, Erlbaum 396–402
64. Schunn, C.D. et al. (2001) Awareness and working memory in 90. Macias, C. et al. (2015) Memory strategically encodes externally
strategy adaptivity. Mem. Cognition 29, 254–266 unavailable information. In Proceedings of the 37th Annual Meet-
ing of the Cognitive Science Society (Noelle, D.C. et al., eds), pp.
65. Gardony, A.L. et al. (2015) Navigational aids and spatial memory
1458–1463, Cognitive Science Society
impairment: the role of divided attention. Spatial Cognition &
Computation 15, 246–284 91. van Nimwegen, C. and van Oostendorp, H. (2009) The question-
able impact of an assisting interface on performance in transfer
66. Gardony, A.L. et al. (2013) How navigational aids impair spatial
situations. Int. J. Ind. Ergonom. 39, 501–508
memory: evidence for divided attention. Spatial Cognition & Com-
putation 13, 319–350 92. Norman, D.A. (2013) The Design of Everyday Things: Revised and
Expanded Edition, Basic books
67. Fenech, E.P. et al. (2010) The effects of acoustic turn-by-turn
navigation on wayfinding. Proc. Hum. Fact. Ergon. Soc. Annu. 93. Parasuraman, R. et al. (1993) Performance consequences of
Meet. 54, 1926–1930 automation-induced ‘complacency’. Int. J. Aviat. Psychol. 3, 1–23
68. Aporta, C. and Higgs, E. (2005) Satellite culture: global positioning 94. Parasuraman, R. and Manzey, D.H. (2010) Complacency and bias
systems, Inuit wayfinding, and the need for a new account of in human use of automation: an attentional integration. Hum.
technology. Curr. Anthropol. 46, 729–753 Factors 52, 381–410

69. Schryer, E. and Ross, M. (2013) The use and benefits of external 95. Wickens, C.D. et al. (2015) Using modeling and simulation to
memory aids in older and younger adults. Appl. Cognitive Psych. predict operator performance and automation-induced compla-
27, 663–671 cency with robotic automation: a case study and empirical valida-
tion. Hum. Factors 57, 959–975
70. Roche, N.L. et al. (2002) Self-awareness of prospective mem-
ory failure in adults with traumatic brain injury. Brain Inj. 16, 96. Ebbatson, M. et al. (2010) The relationship between manual han-
931–945 dling performance and recent flying experience in air transport
pilots. Ergonomics 53, 268–277
71. Knight, R.G. et al. (2005) The effects of traumatic brain injury on the
predicted and actual performance of a test of prospective remem- 97. Casner, S.M. et al. (2014) The retention of manual flying skills
bering. Brain Inj. 19, 19–27 among airline pilots. Hum. Factors 56, 1506–1516

Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9 687


98. Kluge, A. and Frank, B. (2014) Counteracting skill decay: four 99. De Winter, J.C. et al. (2014) Effects of adaptive cruise control and
refresher interventions and their effect on skill and knowledge highly automated driving on workload and situation awareness: a
retention in a simulated process control task. Ergonomics 57, review of the empirical evidence. Transportation research part F:
175–190 traffic psychology and behaviour 27, 196–217

688 Trends in Cognitive Sciences, September 2016, Vol. 20, No. 9

You might also like