DEEP LEARNING-
INTRODUCTION
Umarani Jayaraman
Outline
What is AI?
Machine Learning vs Deep Learning
Types of learning algorithms
Types of deep learning
What is AI?
AI - Make Computers Think and
Behave like Humans
What tasks are easy for a computer?
What tasks are easy for a computer?
AI- Make Computers Think and
Behave like Humans
Formal and Mathematical rules -
Easy for Computers
Example?
What tasks are easy for a computer?
AI - Make Computers Think and
Behave like Humans
Formal and Mathematical rules -
Easy for Computers
Example?
Solving mathematical equations,
addition, multiplication and similar
calculations
What tasks are easy for a
Humans?
AI - Make Computers Think and Behave
like Humans
What tasks are easy for a
Humans?
AI - Make Computers Think and
Behave like Humans
Intuitive Problems for Humans
Example?
What tasks are easy for a
Humans?
AI - Make Computers Think and
Behave like Humans
Intuitive Problems for Humans
Example?
Speech, Text, Visual Recognition, etc.
What tasks are easy for a
Humans?
AI - Make Computers Think and
Behave like Humans
Intuitive Problems for Humans
Example?
Speech, Text, Visual Recognition, etc.
Easy for Humans but difficult for
computers
What tasks are easy for a
Humans?
AI - Make Computers Think and
Behave like Humans
Intuitive Problems for Humans
Example?
Speech, Text, Visual Recognition, etc.
Easy for Humans but difficult for
computers
What tasks are easy for a
Humans?
AI - Make Computers Think and
Behave like Humans
Intuitive Problems for Humans
Example?
Speech, Text, Visual Recognition, etc.
Easy for Humans but difficult for
computers
Intuitive Problems - Complex
concepts can be broken to simple
What tasks are easy for a
Humans?
Intuitive Problems - Complex
concepts can be broken to simple
concepts.
Example: An Image(Complex for
computers) can be broken into
edges, intensity values, objects (like
face, nose, eyes), contours, and
corners.
Relation between AI and other types of
learning
What is AI, ML and deep
15
learning?
Artificial Intelligence- Any technique that
enables computer to mimic human
behavior
Machine Learning
Hand crafted features
Ability to learn without explicitly being
programmed
Deep Learning
Features extraction is built in it
Extract patterns from data using neural
networks
Machine learning vs deep
learning
What is deep learning
Deep Learning definition =>
Subjective
Deep Learning Definition: Computational Graph (Step)
Deep Learning Definition: Hierarchical Relation
What is the goal of deep
learning
Deep Learning definition =>
Subjective.
Learn: Gather knowledge from past
Experience or data.
Deep Learning: Gather knowledge
using simple hierarchical layer
representation of a complex concept
from past Experiences.
Goal: Solve intuitive problems and
reduce, or remove human
Why deep learning?
21
Hand crafted (or) engineering features
are time consuming, hard and not
scalable in practice
Is it possible to learn the underlying
features directly from data?
Why now deep learning
22
trending?
Neural networks evolved in the year
1952
Learning algorithm in machine learning
Learning algorithm in deep learning
Supervised learning
Unsupervised learning
Reinforcement learning
Types of deep learning
Types of deep learning based on direction
of data flow
Feed forward neural network
Recurrent neural network
Feed Forward Neural Networks - Example Networks
Feed Forward Neural Network
Perceptron and Multi-Layer Perceptron(MLP), Fully connected Neural
Network or Artificial Neural Network (ANN)
Convolutional Neural Network
AlexNet
VGG16 and VGG19
ResNet and DenseNet
InceptionNet and GoogleNet
Squeeze & ExcitationNet (SENet)
MobileNet, EfficientNet and Normalizer Free Network (NF Nets)
Generative Adversarial Network (GANs)
Transformers
Bidirectional Encoder Representations from Transformers (BERT)
Generative Pretrained Transformers (GPT - 1, 2, 3, ..)
Vision Transformers
Capsule Network
Recurrent Neural Networks - Example Networks
Recurrent Neural Networks (RNN)
Long Short-Term Memory (LSTM)
Gated Recurrent Unit (GRU)
Fully Recurrent Networks
Hopfield RNN
Neural Turing Machines
Differentiable Neural Computers
Bidirectional RNN
Echo State RNN
Continuous Time RNN
Recursive RNN
Neural History Compressor
Neural Network Pushdown Automata
Memristive Networks
History of deep learning
Source: https://drive.google.com/file/d/1K9XXCJGH_IXLCJ3LgYgsrP4614q2Kxbx/view?usp=sharing
History of deep learning
Source: https://bit.ly/35xXLwu
Chapter1- Deep Learning by Ian Goodfellow
Top researchers in deep
learning
Yoshua Bengio, professor, Université de Montreal,
Quebec, Canada.
Turing Award
Bengio received the 2018 ACM A.M. Turing Award
(often referred to as the "Nobel Prize of
Computing"), together with Geoffrey Hinton and
Yann LeCun, for their work on deep learning.
Bengio, Hinton, and LeCun are sometimes
referred to as the "Godfathers of AI" and
"Godfathers of Deep Learning".
https://yoshuabengio.org/profile/#:~:text=He%20
is%20Co%2DDirector%20of,world%20(h%2Dindex
).
https://analyticsindiamag.com/top-10-de
ep-learning-researchers-who-are-re-defin
ing-its-application-areas
/
Thank you
