Download as PDF, PPTX
![Convolution
𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 1 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛:
𝑦 𝑛 =
𝑘=−∞
𝑘=∞
𝑥 𝑘 ℎ[𝑛 − 𝑘]
𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 2 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑠:
𝑦 𝑛1, 𝑛2 =
𝑘1=−∞
𝑘1=−∞
𝑘2=−∞
𝑘2=∞
𝑥 𝑘1, 𝑘2 ℎ[ 𝑛1 − 𝑘1 , 𝑛2 − 𝑘2 ]](https://image.slidesharecdn.com/cnnbasics-171028092801/85/Overview-of-Convolutional-Neural-Networks-9-320.jpg)
![Convolution
𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 1 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛:
𝑦 𝑛 =
𝑘=−∞
𝑘=∞
𝑥 𝑘 ℎ[𝑛 − 𝑘]
𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 2 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑠:
𝑦 𝑛1, 𝑛2 =
𝑘1=−∞
𝑘1=−∞
𝑘2=−∞
𝑘2=∞
𝑥 𝑘1, 𝑘2 ℎ[ 𝑛1 − 𝑘1 , 𝑛2 − 𝑘2 ]](https://image.slidesharecdn.com/cnnbasics-171028092801/75/Overview-of-Convolutional-Neural-Networks-9-2048.jpg)
More Related Content
Similar to Overview of Convolutional Neural Networks
Overview of Convolutional Neural Networks
- 1. Convolutional Neural Networks Anantharaman PalacodeNarayana Iyer narayana dot Anantharaman at gmail dot com 5 Aug 2017
- 2.
- 3. “A dramatic momentin the meteoric rise of deep learning came when a convolutional network won this challenge for the first time and by a wide margin, bringing down the state-of-the-art top-5 error rate from 26.1% to 15.3% (Krizhevsky et al., 2012), meaning that the convolutional network produces a ranked list of possible categories for each image and the correct category appeared in the first five entries of this list for all but 15.3% of the test examples. Since then, these competitions are consistently won by deep convolutional nets, and as of this writing, advances in deep learning have brought the latest top-5 error rate in this contest down to 3.6%” – Ref: Deep Learning Book by Y Bengio et al
- 4. What is aconvolutional neural network? Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers. • Convolution is a mathematical operation having a linear form
- 5. Types of inputs •Inputs have a structure • Color images are three dimensional and so have a volume • Time domain speech signals are 1-d while the frequency domain representations (e.g. MFCC vectors) take a 2d form. They can also be looked at as a time sequence. • Medical images (such as CT/MR/etc) are multidimensional • Videos have the additional temporal dimension compared to stationary images • Speech signals can be modelled as 2 dimensional • Variable length sequences and time series data are again multidimensional • Hence it makes sense to model them as tensors instead of vectors. • The classifier then needs to accept a tensor as input and perform the necessary machine learning task. In the case of an image, this tensor represents a volume.
- 6. CNNs are everywhere •Image retrieval • Detection • Self driving cars • Semantic segmentation • Face recognition (FB tagging) • Pose estimation • Detect diseases • Speech Recognition • Text processing • Analysing satellite data Copyright 2016 JNResearch, All Rights Reserved
- 7. CNNs for applicationsthat involve images • Why CNNs are more suitable to process images? • Pixels in an image correlate to each other. However, nearby pixels correlate stronger and distant pixels don’t influence much • Local features are important: Local Receptive Fields • Affine transformations: The class of an image doesn’t change with translation. We can build a feature detector that can look for a particular feature (e.g. an edge) anywhere in the image plane by moving across. A convolutional layer may have several such filters constituting the depth dimension of the layer.
- 8. Fully connected layers •Fully connected layers (such as the hidden layers of a traditional neural network) are agnostic to the structure of the input • They take inputs as vectors and generate an output vector • There is no requirement to share parameters unless forced upon in specific architectures. This blows up the number of parameters as the input and/or output dimensions increase. • Suppose we are to perform classification on an image of 100x100x3 dimensions. • If we implement using a feed forward neural network that has an input, hidden and an output layer, where: hidden units (nh) = 1000, output classes = 10 : • Input layer = 10k pixels * 3 = 30k, weight matrix for hidden to input layer = 1k * 30k = 30 M and output layer matrix size = 10 * 1000 = 10k • We may handle this is by extracting the features using pre processing and presenting a lower dimensional input to the Neural Network. But this requires expert engineered features and hence domain knowledge
- 9. Convolution 𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 1𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛: 𝑦 𝑛 = 𝑘=−∞ 𝑘=∞ 𝑥 𝑘 ℎ[𝑛 − 𝑘] 𝐶𝑜𝑛𝑣𝑜𝑙𝑢𝑡𝑖𝑜𝑛 𝑖𝑛 2 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑠: 𝑦 𝑛1, 𝑛2 = 𝑘1=−∞ 𝑘1=−∞ 𝑘2=−∞ 𝑘2=∞ 𝑥 𝑘1, 𝑘2 ℎ[ 𝑛1 − 𝑘1 , 𝑛2 − 𝑘2 ]
- 11. CNNs Types of layersin a CNN: • Convolution Layer • Pooling Layer • Fully Connected Layer
- 12. Convolution Layer • Alayer in a regular neural network take vector as input and output a vector. • A convolution layer takes a tensor (3d volume for RGB images) as input and generates a tensor as output Fig Credit: Lex Fridman, MIT, 6.S094
- 13. Slide Credit: LexFridman, MIT, 6.S094
- 14. Local Receptive Fields •Filter (Kernel) is applied on the input image like a moving window along width and height • The depth of a filter matches that of the input. • For each position of the filter, the dot product of filter and the input are computed (Activation) • The 2d arrangement of these activations is called an activation map. • The number of such filters constitute the depth of the convolution layer Fig Credit: Lex Fridman, MIT, 6.S094
- 15. Convolution Operation betweenfilter and image • The convolution layer computes dot products between the filter and a piece of image as it slides along the image • The step size of slide is called stride • Without any padding, the convolution process decreases the spatial dimensions of the output Fig Credit: A Karpathy, CS231n
- 16. Activation Maps • Example: •Consider an image 32 x 32 x 3 and a 5 x 5 x 3 filter. • The convolution happens between a 5 x 5 x 3 chunk of the image with the filter: 𝑤 𝑇 𝑥 + 𝑏 • In this example we get 75 dimensional vector and a bias term • In this example, with a stride of 1, we get 28 x 28 x 1 activation for 1 filter without padding • If we have 6 filters, we would get 28 x 28 x 6 without padding • In the above example we have an activation map of 28 x 28 per filter. • Activation maps are feature inputs to the subsequent layer of the network • Without any padding, the 2D surface area of the activation map is smaller than the input surface area for a stride of >= 1 Copyright 2016 JNResearch, All Rights Reserved
- 17. Stacking Convolution Layers FigCredit: A Karpathy, CS231n
- 18. Feature Representation asa hierarchy
- 19. Padding • The spatial(x, y) extent of the output produced by the convolutional layer is less than the respective dimensions of the input (except for the special case of 1 x 1 filter with a stride 1). • As we add more layers and use larger strides, the output surface dimensions keep reducing and this may impact the accuracy. • Often, we may want to preserve the spatial extent during the initial layers and downsample them at a later time. • Padding the input with suitable values (padding with zero is common) helps to preserve the spatial size
- 20. Zero Padding theborder Fig Credit: A Karpathy, CS231n
- 21. Hyperparameters of theconvolution layer • Filter Size • # Filters • Stride • Padding Fig Credit: A Karpathy, CS231n
- 22. Pooling Layer • Poolingis a downsampling operation • The rationale is that the “meaning” embedded in a piece of image can be captured using a small subset of “important” pixels • Max pooling and average pooling are the two most common operations • Pooling layer doesn’t have any trainable parameters Fig Credit: A Karpathy, CS231n
- 23.
- 24.
- 25. Current trend: DeeperModels • CNNs consistently outperform other approaches for the core tasks of CV • Deeper models work better • Increasing the number of parameters in layers of CNN without increasing their depth is not effective at increasing test set performance. • Shallow models overfit at around 20 million parameters while deep ones can benefit from having over 60 million. • Key insight: Model performs better when it is architected to reflect composition of simpler functions than a single complex function. This may also be explained off viewing the computation as a chain of dependencies
- 27.
- 28.
- 29.
- 30. Core Tasks ofComputer Vision Core CV Task Task Description Output Metrics Classification Given an image, assign a label Class Label Accuracy Localization Determine the bounding box containing the object in the given image Box given by (x1, y1, x2, y2) Ratio of intersection to the union (Overlap) between the ground truth and bounding box Object Detection Given an image, detect all the objects and their locations in the image For each object: (Label, Box) Mean Avg Best Overlap (MABO,) mean Average Precision (mAP) Semantic Segmentation Given an image, assign each pixel to a class label, so that we can look at the image as a set of labelled segments A set of image segments Classification metrics, Intersection by Union overlap Instance Segmentation Same as semantic segmentation, but each instance of a segment class is determined uniquely A set of image segments
- 31. Object Localization • Givenan image containing an object of interest, determine the bounding box for the object • Classify the object
- 32. Slide Credit: AKarpathy, CS231n
- 34. Slide Credit: AKarpathy, CS231n
- 43. Datasets for evaluation •Imagenet challenges provide a platform for researchers to benchmark their novel algorithms • PASCAL VOC 2010 is great for small scale experiments. About 1.3 GB download size. • MS COCO datasets are available for tasks like Image Captioning. Download size is huge but selective download is possible.