0% found this document useful (0 votes)

18 views29 pages

Introduction ANN

The lecture notes cover the fundamentals of artificial neural networks and deep learning, focusing on biological neurons and the perceptron model. Key concepts include the structure of neurons, the perceptron learning algorithm, and the need for multilayer perceptrons due to limitations in simple perceptrons. The document also discusses activation functions, feedforward neural networks, and training techniques such as backpropagation.

Uploaded by

nguyenvothanhdatcbt10l

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

18 views29 pages

Introduction ANN

Uploaded by

nguyenvothanhdatcbt10l

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 29

Lecture Notes on Applied Aritficial Intelligence (ME-3181)

Introduction to Artificial Neural

Networks and Deep Learning
Phung Thanh Huy

Ho Chi Minh City Univerisyt of Technology

Vietnam National University Ho Chi Minh City

Ho Chi Minh City, 2023

Biological Neurons (Peceptrons)

• Axon: Sợi trục

• Nucles: Nhân
• Dendrite: Sợi nhánh
• Cell body: Soma thần kinh
• Synaptic terminal: Cúc xi
náp

- Each neuron typically connected to thousands of other neurons

- The signals are sent via dendrites to the cell body, the strength of the connections
(dendrites) are different.
- When a neuron receives a sufficient amount of these neurotransmitters within a
few milliseconds, it fires its own

Applied Artificial Intelligence (ME-3181) Pg. 2

Biological Neurons (Peceptrons)

- Warren McCulloch and Walter Pitts first describe the working of the neurons in
1940s.
- The McCulloch-Pitts neuron model was implemented in 1950s.

Credits: S.I. Lee, UST – ETRI

Credits: Sebastian - Caltech
Applied Artificial Intelligence (ME-3181) Pg. 3
Rosenblatt’s Peceptron

+1, 𝑖𝑓 ≥ 0
𝑓 = 𝑠𝑖𝑔𝑛 𝑧 = ቊ
−1, 𝑖𝑓 𝑧 < 0

z Class? Or
f
+1, 𝑖𝑓 ≥ 0
𝑓 = 𝑠𝑡𝑒𝑝 𝑧 = ቊ
0, 𝑖𝑓 𝑧 < 0

𝑤𝑖 : weigths
𝑧 = 𝑤0 𝑥0 + 𝑤1 𝑥1 + ⋯ + 𝑤𝑛 𝑥𝑛 = 𝑤 𝑇 𝑥
𝑤0 : bias
𝑦 = 𝑓 𝑧 = 𝑓 𝑤𝑇𝑥
𝑥0 = 1

Applied Artificial Intelligence (ME-3181) Pg. 4

Rosenblatt’s Peceptron

Perceptron Learning algorithm:

𝑤 𝑛𝑒𝑥𝑡 = 𝑤 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 + 𝜂 𝑦𝑖 − 𝑦ො𝑖 𝑥𝑖 (1)

𝑤 𝑛𝑒𝑥𝑡 = 𝑤 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 + 𝜂𝑦𝑖 𝑥𝑖 (2)
Step 1: Initialize 𝑤 0 Missed Classified
Step 2: Calculate 𝑦ො = 𝑓 𝑧
Step 3: Check whether 𝑦𝑖 is miss-classified
Step 4: Update 𝑤 𝑛𝑒𝑥𝑡

Exercise
1. Prove that Eqs. (1) and (2) are equivalent
2. Implement Perceptron Learning Algorithm and use it for Iris dataset
Applied Artificial Intelligence (ME-3181) Pg. 5
Peceptron and logical classification

Applied Artificial Intelligence (ME-3181) Pg. 6

Peceptron and logical classification
AND and OR

Applied Artificial Intelligence (ME-3181) Pg. 7

Peceptron and logical classification
Issue with XOR
Origin of Multiple Perceptron!

Applied Artificial Intelligence (ME-3181) Pg. 8

Convergence of a perceptron
• Rosenblatt proved mathematically that the perceptron learning rule
converges if the two classes can be separated by a linear hyperplane.
• If the classes cannot be separated perfectly by such a linear decision
boundary, the weights will never stop updating unless we set a
maximum number of epochs.

Applied Artificial Intelligence (ME-3181) Pg. 9

Need for multiple units and multiple layers

Applied Artificial Intelligence (ME-3181) Pg. 10

Multilayer-peceptron – Artificial Neural Networks (ANN)
Multilayer Perceptron

(Scikit Learn uses Multi-layer perceptron)

Fully Connected Network

Applied Artificial Intelligence (ME-3181) Pg. 11

Activation functions
Popular activation functions

Applied Artificial Intelligence (ME-3181) Pg. 12

Activation functions
For outputs
• Regression:
No active function
• Classification
Multiclass: softmax

Applied Artificial Intelligence (ME-3181) Pg. 13

Feedforward Neural Networks
Feedforward neural network

Flow: Input → Ouput

𝑗
𝑤𝑖 are known