Downloaded 170 times
![Activation Function
STEP Function
• If value of X is greater than or equal to 0, then output is 1,
If value of X is less than 0, then output is 0
• NN uses back propagation & Gradient descent method to
calculate weight of different layers
• Since step Function is non differentiable to zero, it can’t do
the gradient descent method, so it can’t update weights.
SIGMOID Function
• If value of X is infinity, then output is 1,
If value of X is negative infinity, then output is 0
• It captures non-linearity in the data
• It can use Gradient descent & Back propagation method to
calculate weights.
• Output range [0,1]
Image source from Towards Data Science
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/85/Development-of-Deep-Learning-Architecture-14-320.jpg)
![Activation Function
Tanh Function
• Rescaled of Sigmoid Function
• Output range [-1,1]
• Better learning rate requires, higher gradient. In some times,
for the data is centred around 0, derivatives are higher.
ReLU Function
• Rectified Linear Unit. if any negative input is detected, it
returns 0, otherwise it returns the value back.
Leaky ReLU Function
• Same as ReLU, like returning same for Positive. But for
negative values instead of returning zero, it has constant slope
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/85/Development-of-Deep-Learning-Architecture-15-320.jpg)
![In Tensor Flow
Y =
tf.nn.softmax(tf.matmul(X,W)+b)
W[784x10]
X[100x784]
b[10]
Y[100x10]
Cross Entropy
0 0 1 0 0 0 0 0 00
0.2 0.9 0.5 0.3 0.1 0.2 0.1 0.3 0.10.1
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
Actual Probability
Computed Probability
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/85/Development-of-Deep-Learning-Architecture-25-320.jpg)
![Activation Function
STEP Function
• If value of X is greater than or equal to 0, then output is 1,
If value of X is less than 0, then output is 0
• NN uses back propagation & Gradient descent method to
calculate weight of different layers
• Since step Function is non differentiable to zero, it can’t do
the gradient descent method, so it can’t update weights.
SIGMOID Function
• If value of X is infinity, then output is 1,
If value of X is negative infinity, then output is 0
• It captures non-linearity in the data
• It can use Gradient descent & Back propagation method to
calculate weights.
• Output range [0,1]
Image source from Towards Data Science
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/75/Development-of-Deep-Learning-Architecture-14-2048.jpg)
![Activation Function
Tanh Function
• Rescaled of Sigmoid Function
• Output range [-1,1]
• Better learning rate requires, higher gradient. In some times,
for the data is centred around 0, derivatives are higher.
ReLU Function
• Rectified Linear Unit. if any negative input is detected, it
returns 0, otherwise it returns the value back.
Leaky ReLU Function
• Same as ReLU, like returning same for Positive. But for
negative values instead of returning zero, it has constant slope
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/75/Development-of-Deep-Learning-Architecture-15-2048.jpg)
![In Tensor Flow
Y =
tf.nn.softmax(tf.matmul(X,W)+b)
W[784x10]
X[100x784]
b[10]
Y[100x10]
Cross Entropy
0 0 1 0 0 0 0 0 00
0.2 0.9 0.5 0.3 0.1 0.2 0.1 0.3 0.10.1
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
Actual Probability
Computed Probability
www.pantechsolutions.net](https://image.slidesharecdn.com/pantechdeeplearning-200525045728/75/Development-of-Deep-Learning-Architecture-25-2048.jpg)
Uploaded byPantech ProLabs India Pvt Ltd
Development of Deep Learning Architecture
This document provides information about a development deep learning architecture event organized by Pantech Solutions and The Institution of Electronics and Telecommunication. The event agenda includes general talks on AI, deep learning libraries, deep learning algorithms like ANN, RNN and CNN, and demonstrations of character recognition and emotion recognition. Details are provided about the organizers Pantech Solutions and IETE, as well as deep learning topics like neural networks, activation functions, common deep learning libraries, algorithms, applications, and the event agenda.
More Related Content
Similar to Development of Deep Learning Architecture
More from Pantech ProLabs India Pvt Ltd
Development of Deep Learning Architecture
- 1. Development of Deep LearningArchitecture Organized by Pantech Solutions & The Institution of Electronics and Telecommunication
- 2. All Participants, Pleasefill the Attendance form, Link given in the description Instruction for Attendees www.pantechsolutions.net
- 3. Founded in 1953|1,25,000members | 64 Centers Profile IETE conducts Technical Meetings | Conferences | Symposia | Exhibitions | delivering career advancement opportunity and publishes Technical journals IETE Focus on Computer Science & Engineering and Information Technology streams and Diploma DIPIETE Exams| ALCCS Post graduate source IETE Exams About IETE www.pantechsolutions.net
- 4. Established on 2004| 7 Branches | 100+ Team Profile Manufacturer of Lab equipment‘s & Development boards | Industrial &Funded projects | Online retail store of engineering products and projects R & D Lab equipment, Engineering Kits, Components, Sensors and All level Projects Online Retail Store 1500+ Workshops | 250+ FDP | 100+ Seminars Training About Pantech www.pantechsolutions.net
- 5. “To Gain Global Leadershipin providing Technological Solutions through Sustained Innovation” Pantech Technology A I Machine Learning | Deep Learning | NLP | Chatbot | Block chain | Data science and AI boards S o f t w a r e AR & VR products | Android development | Cyber security | Cloud & Mobile computing | IoT B r a i n C o m p u t e r I n t e r f a c e Brain wave analysis | Brain controlled applications | EEG headband M e c h a n i c a l Robotics | Mechanical components | 3D design and printing | E l e c t r i c a l E- Vehicle | Renewable energy systems | Motors and Drivers |Power electronics | Power Systems E l e c t r o n i c s Manufacturer of Embedded development Boards | FPGA Boards | Sensors | Interface Modules www.pantechsolutions.net
- 6. Agenda 01 General talk onAI 02 About Deep Learning & Libraries 03 Deep Learning Algorithm ANN | RNN | CNN 04 Application of Deep Learning 05 Demo on Character recognition & Emotion Recognition www.pantechsolutions.net
- 7. What is AI Intelligenceis given by feeding the experience of human to machines in the form of data Intelligence demonstrated by the Machines in contrast to Natural Intelligence delivered by Humans “AI WILL BE THE BEST OR WORST THING EVER FOR HUMANITY.” – ELON MUSK “SUCCESS IN CREATING AI WOULD BE THE BIGGEST EVENT IN HUMAN HISTORY. UNFORTUNATELY, IT MIGHT ALSO BE THE LAST, UNLESS WE LEARN HOW TO AVOID THE RISKS.” – STEPHEN HAWKING www.pantechsolutions.net
- 8.
- 9. Deep Learning Subset ofMachine learning, Algorithms inspired by the structure and function of Human Brain Brain Vision Recognition Dataset Model Recognition Training www.pantechsolutions.net
- 10. Why Deep Learning? Input Data Its an Apple Prediction Model AI Automatic Feature Extraction www.pantechsolutions.net
- 11. Why GPU High Bandwidth| Thread Parallelism | Easily programmable registers | Bandwidth Optimized GPU – Graphical Processing Unit 100% 50%Less Bandwidth | Latency Optimised | Not suitable for Real time application where performance is the main Factor CPU –Central Processing Unit “It Can train more number of data in short time period & Better for real time AI based application” www.pantechsolutions.net
- 12. Deep Learning Libraries? Numerical computation using data flow graphs | Backend for Keras | Distributed computing on multiple GPU Tensor Flow Framework for Deep learning | Same code for CPU & GPU | Uses Theano/TF as Backend | CNN | not support multi GPU Keras Very high performance | Tuning Hyper parameters | CNN & RNN(Facebook extends) | | CPU & GPU Caffe Old ML & DL library | Supports CUDA for parallel computation | Supervised image problem with CNN Torch Basic operation such as Sorting, Reshaping, indexing | Scientific computing | Most new feature belong in SciPy rather than Numpy Numerical Python & Scientific Python More number of Language Bindings | Distributed computing Mxnet www.pantechsolutions.net
- 13. Neuron & ActivationFn. • Like a Human Brain, here a Neuron takes input and do some function to give the output • Function going to be the Mathematical function • Those Function is known as ACTIVATION function Neuron Activation Function • Step Function • Sigmoid Function • Tanh Function • ReLU Function Dendrites Cell Body Axon Synapse Neuron OUTPUT Activation Fn. Inputs www.pantechsolutions.net
- 14. Activation Function STEP Function •If value of X is greater than or equal to 0, then output is 1, If value of X is less than 0, then output is 0 • NN uses back propagation & Gradient descent method to calculate weight of different layers • Since step Function is non differentiable to zero, it can’t do the gradient descent method, so it can’t update weights. SIGMOID Function • If value of X is infinity, then output is 1, If value of X is negative infinity, then output is 0 • It captures non-linearity in the data • It can use Gradient descent & Back propagation method to calculate weights. • Output range [0,1] Image source from Towards Data Science www.pantechsolutions.net
- 15. Activation Function Tanh Function •Rescaled of Sigmoid Function • Output range [-1,1] • Better learning rate requires, higher gradient. In some times, for the data is centred around 0, derivatives are higher. ReLU Function • Rectified Linear Unit. if any negative input is detected, it returns 0, otherwise it returns the value back. Leaky ReLU Function • Same as ReLU, like returning same for Positive. But for negative values instead of returning zero, it has constant slope www.pantechsolutions.net
- 16. Activation Function SOFTMAX Function •Softmax activation function will be applied in the last layer of Neural network, instead of ReLU, tanh, Sigmoid. • It is used to map the non-normalized output of a network to a probability distribution over predicted output class. That is it converts output of last layer into a essential probability distribution. www.pantechsolutions.net
- 17. Deep Learning Algorithm ARTIFICIALNEURAL NETWORK (ANN)01 RECURRENT NEURAL NETWORK (ANN)02 CONVOLUTIONAL NEURAL NETWORK (ANN)03 www.pantechsolutions.net
- 18. Vanishing & ExplodingGradient • It is very common problem in every Neural Network, which is associated with Backpropagation. • Weights of network are updated through backpropagation by finding gradients. • When the number of hidden layer is high, then the gradient vanishes or explodes as it propagates backward. It leads instability in network, unable to learn from training • The explosion occurs through exponential growth by repeatedly multiplying gradients through the network layers that have values larger than 1.0 • It can be fixed by redesigning the network, using Long Short Term Memory networks, Gradient clipping, etc.
- 19. ANN • Learns anyNon-Linear Function, It is known as Universal Function Approximators • Activation Function introduce non linear property to network, so it will identify complex relationship between input & output • Output of each neuron is the activation of weighted sum of Input, If there is no Activation function, network can't learn non-linear function • Feed Forward Neural Network – Input processed in one direction, When hidden layer is more than one, that is Deep Neural Network Input Hidden Output www.pantechsolutions.net
- 20. RNN • Looping systemin hidden layer of ANN is known as RNN • It captures sequential info of input data, that is dependency between words to make prediction. Whereas, ANN cannot capture sequential information • RNN shares parameters across different time steps, so that there will be few parameter to train • It is the time series version of ANN. Common Recurrent layers are LSTM(Long Short Term Memory) & GRU (Grated Recurrent Units) • GRU is used to how much pass data needed to flow through model • It is mostly used in NLP (Natural Language Processing) Input Hidden Output www.pantechsolutions.net
- 21. CNN • CNN learnsthe filter automatically to extract the right features from the data • It captures spatial features (Arrangement of pixels) whereas ANN can’t. • It also follows parameter sharing like RNN, applies single filter in different part of single image. Whereas ANN can’t. • It don’t have recurrent connections like RNN, instead it has convolution type of hidden layers • Convolution and pooling functions are used as activation functions • CONVOLUTION: Input image and other as Filter on input image(Kernel) produces output image. • POOLING: picking maximum value from selected region is Max pooling and vice versa. www.pantechsolutions.net
- 22. CNN Architecture n3 units Output . . . . . . . . . . . . . Flattened n2channels (4 x 4 x n2) n2 channels (8 x 8 x n2) n1 channels (12 x 12 x n1) n1 channels (24 x 24 x n1) Input 28x28x1 0 1 2 9 Conv_1 Convolution (5x5) Max-pooling (2x2) Conv_2 Convolution (5x5) Max-pooling (2x2) FC_3 Fully Connected ReLU Activation FC_4 Fully Connected Neural Network www.pantechsolutions.net
- 23. Simple Softmax Classification Input 28x28x1 784Pixels …. . . . . . . . . 0 1 3 92 www.pantechsolutions.net
- 24. 100 image ata time … . … . … . . . . … . 100x784 (100 images Flattened) W0,0 W0,1 W0,2 ……W0,9 W1,0 W1,1 W1,2 ……W1,9 W2,0 W2,1 W2,2 ……W2,9 W3,0 W3,1 W3,2 ……W3,9 W4,0 W4,1 W4,2 ……W4,9 . . . W783,0 W783,1 W783,2 ……W783,9 784x10 Matrix Input Image Weights L0,0 W0,1 W0,2 ……W0,9 L1,0 W1,1 W1,2 ……W1,9 L2,0 W2,1 W2,2 ……W2,9 L3,0 W3,1 W3,2 ……W3,9 L4,0 W4,1 W4,2 ……W4,9 . . . L99,0 L99,1 L99,2 ……W99,9 X X + b0 b1 b2 b3 ….. b9 1x10 Matrix Bias 100x10 Matrix www.pantechsolutions.net
- 25. In Tensor Flow Y= tf.nn.softmax(tf.matmul(X,W)+b) W[784x10] X[100x784] b[10] Y[100x10] Cross Entropy 0 0 1 0 0 0 0 0 00 0.2 0.9 0.5 0.3 0.1 0.2 0.1 0.3 0.10.1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Actual Probability Computed Probability www.pantechsolutions.net
- 26. Application of DeepLearning It is kind of machine analyses the symptoms and cause and suggesting the Medicine for the patient. Those knowledge is fed by tons of Medical data, based on the applications. AI Doctor It is the application uses medical images, to classify the diseased and healthy like Diabetic Retinopathy Prediction of Disease from Medical Image ALS (amyotrophic lateral sclerosis) is a kind of disease which causes speech problems, the voice of the patients completely non understandable. AI helps to train the voice of ALP patients to deliver voice based assistance for them. Voice recognition for ALS Patient www.pantechsolutions.net
- 27. Application of DeepLearning After cultivation, Fruits and Vegetables needed to be segregated based on the quality like Rotten or Healthy. AI application will done this with great accuracy Fruit & vegetable classification AI in Autonomous Vehicle Now AI is used for studying and identifying the pattern of molecular structure of Food Items to deliver a new dish recipes. AI in Cooking AI in voice Assistance
- 28.
- 29.
- 30.
- 31. THANK YOU May 30– Conversational Bot design April 2 – Brain Computer Interface Upcoming Session