Downloaded 26 times
Uploaded byKobkrit Viriyayudhakorn
[Lecture 4] AI and Deep Learning: Neural Network (Theory)
This lecture discusses shallow neural networks. It introduces neural network concepts like network representation, vectorizing training examples, and activation functions such as sigmoid, tanh, and ReLU. It explains that activation functions introduce non-linearity crucial for neural networks. The lecture also covers backpropagation for computing gradients in neural networks and updating weights with gradient descent. Random initialization is discussed as important for breaking symmetry in networks.
More Related Content
Similar to [Lecture 4] AI and Deep Learning: Neural Network (Theory)
![[Lecture 3] AI and Deep Learning: Logistic Regression (Coding)](https://cdn.slidesharecdn.com/ss_thumbnails/lecture3empty-180216132805-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![[Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://cdn.slidesharecdn.com/ss_thumbnails/lecture2-ink-180216131533-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
[Lecture 4] AI and Deep Learning: Neural Network (Theory)
- 1. Introduction to Artificial Intelligence& Deep Learning Lecture 4: Shallow Neural Network Dr. Kobkrit Viriyayudhakorn
- 2. What is aNeural Network? 𝑧 = 𝑤 𝑇 𝑥 + 𝑏𝑤 𝑏 𝑎 = 𝜎 𝑧 ℒ 𝑎, 𝑦 𝑥 Logistic Regression Neural Network 𝑥 𝑧 1 = 𝑤 1 𝑥 + 𝑏 1𝑤 1 𝑏 1 𝑎 1 = 𝜎 𝑧 1 ℒ 𝑎 2 , 𝑦𝑧 2 = 𝑤 2 𝑎 1 + 𝑏 2 𝑤 2 𝑏 2 𝑎 2 = 𝜎 𝑧 2
- 3.
- 4. Neural Network Representation 𝑤𝑇 𝑥 + 𝑏 𝜎 𝑧 𝑥1 𝑥2 𝑥3 𝑎 = 𝑦
- 5.
- 6. Neural Network RepresentationLearning
- 7. Vectorizing Across MultipleTraining Examples
- 8. Vectorizing Across MultipleTraining Examples
- 9. Explanation for VectorizedImplementation
- 10. Explanation for VectorizedImplementation
- 11.
- 12.
- 13. Activation Function :ReLu and Leaky ReLu
- 14. Pro and consfor Activation functions • Sigmoid
- 15. Pro and consfor Activation functions • Tanh
- 16. Pro and consfor Activation functions • ReLu and Leaky ReLu
- 17. Why Neuron Networkneed the activation function?
- 18. Why Neuron Networkneed the activation function?
- 19. Derivative of Activationfunction • Sigmoid
- 20. Derivative of Activationfunction • Tanh
- 21. Derivative of Activationfunction • ReLu and Leaky ReLu
- 22. Gradient Descent forNeural Network
- 23. Formulas For ComputingDerivatives
- 24. Backpropagation Intuition • LogisticRegression 𝑧 = 𝑤 𝑇 𝑥 + 𝑏𝑤 𝑏 𝑎 = 𝜎 𝑧 ℒ 𝑎, 𝑦 𝑥
- 25. Backpropagation Intuition • NeuralNetwork 𝑥 𝑧 1 = 𝑤 1 𝑥 + 𝑏 1𝑤 1 𝑏 1 𝑎 1 = 𝜎 𝑧 1 ℒ 𝑎 2 , 𝑦𝑧 2 = 𝑤 2 𝑎 1 + 𝑏 2 𝑤 2 𝑏 2 𝑎 2 = 𝜎 𝑧 2
- 26. Summary of GradientDescent ⅆ𝑧 2 = 𝑎 2 − 𝑦 ⅆ𝑤 2 = 𝑑𝑧 2 𝑎 1 𝑇 ⅆ𝑏 2 = ⅆ𝑧 2 ⅆ𝑧 1 = 𝑤 2 𝑇 ⅆ𝑧 2 ∗ g 1 ′ 𝑧 1 ⅆ𝑤 1 = 𝑑𝑧 1 𝑥 𝑇 ⅆ𝑏 1 = ⅆ𝑧 1
- 27.
- 28.