Introduction to
Neural Network
Aplikasi Teknologi Cerdas
2020
Outline
• Overview (Examples)
• History
• Biological Inspiration
• Single Neuron Model
Reference
1. Simon S Haykin, Neural networks - A Comprehensive
Foundation, Prentice Hall (2008)
2. Martin T. Hagan et. al, Neural Network Design
2nd Edition
Face Unlock Technology
How it works?
Crack Detection
Object Recognition
Vehicle Detection
How it works?
Plate Number
Identification
Face Unlock Technology
Vehicle Recognition
Plate Number Identification
Crack Detection
} Neural Network Application
(Image Processing, Pattern Recognition, Classification)
Other application:
• Forecasting
• Intelligent Control
• Online Shop Recommendation
• etc.
History
Prehistory:
• Golgi and Ramon y Cajal study the nervous system and discover neurons (end of
19th century)
History (brief):
• McCulloch and Pitts (1943): the first artificial neural network with binary neurons
• Hebb (1949): learning = neurons that are together wire together
• Minsky (1954): neural networks for reinforcement learning
• Taylor (1956): associative memory
• Rosenblatt (1958): perceptron, a single neuron for supervised learning
• Widrow and Hoff (1960): Adaline
• Minsky and Papert (1969): limitations of single-layer perceptrons (and they
erroneously claimed that the limitations hold for multi-layer perceptrons)
6
History
Stagnation in the 70's:
• Individual researchers continue laying foundations
• von der Marlsburg (1973): competitive learning and self-organization
Big neural-nets boom in the 80's
• Grossberg: adaptive resonance theory (ART)
• Hopfield: Hopfield network
• Kohonen: self-organising map (SOM)
• Oja: neural principal component analysis (PCA)
• Ackley, Hinton and Sejnowski: Boltzmann machine
• Rumelhart, Hinton and Williams: backpropagation
Diversification during the 90's:
• Machine learning: mathematical rigor, Bayesian methods, infomation theory,
• Computational neurosciences: workings of most subsystems of the brain are
understood at some level 7
Biological Inspiration
• Human Brain/ Nervous System
Approximately 10^11
neurons here
Biological Inspiration
Biological Neuron
• Dendrites: carry electrical signals into
the cell body.
• Cell body : effectively sums and
thresholds these incoming signals.
• Axon: carries the signal from the cell
body out to other neurons.
• Synapse: The point of contact between
an axon of one cell and a dendrite of
another cell
Neuron Model
Single Biological Neuron Single Neuron Model
Single Neuron Model
p : input
w : weight
b : bias
f() : activation function
a : output
Neural Network
A NN is specified by
• an architecture: a set of neurons and links connecting neurons. Each link has a weight
• a neuron model: the information processing unit of the NN
• a learning algorithm: used for training the NN by modifying the weights in order to
solve the particular learning task correctly on the training examples
