Dialogues on the role of top-down factors in sensory processing
Does the human brain implement predictive coding?
Karl Friston, University College London
How much about our interaction with – and experience of – our world can be deduced from basic
principles? This talk reviews recent attempts to understand the self-organised behaviour of embodied
agents, like ourselves, as satisfying basic imperatives for sustained exchanges with the environment.
In brief, one simple driving force appears to explain many aspects of action and perception. This
driving force is the minimisation of surprise or prediction error. In the context of perception, this
corresponds to Bayes-optimal predictive coding that suppresses exteroceptive prediction errors.
In the context of action, motor reflexes can be seen as suppressing proprioceptive prediction errors.
We will look at some of the phenomena that emerge from this scheme, such as hierarchical message
passing in the brain and the ensuing perceptual inference.
.
Overview
The anatomy of inference graphical models
canonical microcircuits
Functional asymmetries spectral connections
modulatory connections
Action and perception inference and uncertainty
simulations of saccadic searches
“Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz
Hermann von Helmholtz Richard Gregory
Geoffrey Hinton
The Helmholtz machine and the
Bayesian brain
Thomas Bayes Richard Feynman
“Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz
Hermann von Helmholtz Richard Gregory
sensory impressions…
Plato: The Republic (514a-520a)
Bayesian filtering and predictive coding
prediction update
prediction error
Minimizing prediction error
sensations – predictions
Prediction error
Action Perception
Change sensations Change predictions
Generative models
what where
A simple hierarchy
Sensory
fluctuations
From models to perception
A simple hierarchy
Generative model
Descending
predictions
ModelPredictive
inversion (inference)
coding
Expectations:
Ascending
prediction errors
Predictions:
Prediction errors:
Canonical microcircuits for predictive coding
Haeusler and Maass: Cereb. Cortex 2006;17:149-162 Bastos et al: Neuron 2012; 76:695-711
David Mumford
Predictive coding with reflexes Action
oculomotor
signals
reflex arc
proprioceptive input
pons
retinal input
Perception
Errors (superficial pyramidal cells)
frontal eye fields geniculate
Top-down or backward
predictions
Expectations (deep pyramidal cells)
Bottom-up or forward
prediction error
visual cortex
Overview
The anatomy of inference graphical models
canonical microcircuits
Functional asymmetries spectral connections
modulatory connections
Action and perception inference and uncertainty
simulations of saccadic searches
Errors (superficial pyramidal cells)
Forward transfer function
14
12
10
spectral power
Expectations (deep pyramidal cells) 8
4
Andre Bastos
2
0
0 20 40 60 80 100
0.3 V4 V1
0.25
0.2
0.15 superficial
0.1
0.05
0
0 20 40 60 80 100 120
spectral power
4
2
3
2
deep
1
1
0
0 20 40 60 80 100
frequency (Hz)
0
0 20 40 60 80 100 120
frequency (Hz) Backward transfer function
Errors (superficial pyramidal cells)
Expectations (deep pyramidal cells)
Linear or driving connections
superficial
Nonlinear or modulatory
connections
deep
NMDA receptor density
Interim summary
Hierarchical predictive coding is a neurobiological plausible scheme that the brain
might use for (approximate) Bayesian inference about the causes of sensations
Predictive coding requires the dual encoding of expectations and errors, with
reciprocal (neuronal) message passing
Much of the known neuroanatomy and neurophysiology of cortical architectures is
consistent with the requisite message passing
In particular, the functional asymmetries and laminar specificity of intrinsic and
extrinsic connections provide a formal perspective on spectral asymmetries and
nonlinear coupling in the brain.
Overview
The anatomy of inference graphical models
canonical microcircuits
Functional asymmetries spectral connections
modulatory connections
Action and perception inference and uncertainty
simulations of saccadic searches
Sampling the world to minimise uncertainty
Free energy minimisation Expected uncertainty
Likelihood World model Prior beliefs
“I am [ergodic] therefore I think” “I think therefore I am [ergodic]”
Sampling the world to minimise uncertainty
Free energy minimisation Expected uncertainty
stimulus visual input salience sampling
Perception as hypothesis testing – saccades as experiments
Parietal (where)
Frontal eye fields
Visual cortex
Fusiform (what) Pulvinar salience map
oculomotor reflex arc Superior colliculus
Saccadic eye movements
Saccadic fixation and salience maps
Action (EOG)
2
Hidden (oculomotor) states 0
-2
200 400 600 800 1000 1200 1400
time (ms)
Visual samples
Posterior belief
5
Conditional expectations 0
about hidden (visual) states -5
200 400 600 800 1000 1200 1400
time (ms)
And corresponding percept
Hermann von Helmholtz
“Each movement we make by which we alter the appearance of
objects should be thought of as an experiment designed to test
whether we have understood correctly the invariant relations of
the phenomena before us, that is, their existence in definite
spatial relations.”
‘The Facts of Perception’ (1878) in The Selected Writings of Hermann von
Helmholtz, Ed. R. Karl, Middletown: Wesleyan University Press, 1971 p. 384
Thank you
And thanks to collaborators:
Rick Adams
Andre Bastos
Sven Bestmann
Harriet Brown
Jean Daunizeau
Mark Edwards
Xiaosi Gu
Lee Harrison
Stefan Kiebel
James Kilner
Jérémie Mattout
Rosalyn Moran
Will Penny
Lisa Quattrocki Knight
Klaas Stephan
And colleagues:
Andy Clark
Peter Dayan
Jörn Diedrichsen
Paul Fletcher
Pascal Fries
Geoffrey Hinton
James Hopkins
Jakob Hohwy
Henry Kennedy
Paul Verschure
Florentin Wörgötter
And many others
