Image classification with neural networks

Image classification with neural networks

• 1.
  Amirkabir University ofTechnology Department of Computer Engineering and Information Technology Image Classification with Deep Convolutional Neural Networks Sepehr Rasouli
• 2.
  Introduction > Methods> Results > Conclusion2 Outline • Introduction to Image Classification & Deep Networks • Proposed Method • Main Idea • Data Set • Architecture • Techniques • Comparison & Results • Conclusion
• 3.
  Introduction > Methods> Results > Conclusion3 Image Classification
• 4.
  Introduction > Methods> Results > Conclusion4 Why Deep Learning? •“Shallow” vs. “deep” architectures Learn a feature hierarchy all the way from pixels to classifier Hand Designed Feature Extraction Trainable Classifier Layer 1 Layer N Simpler classifier
• 5.
  Introduction > Methods> Results > Conclusion5 Our Method • Deep Convolutional Neural Network • 5 convolutional and 3 fully connected layers • 650,000 neurons, 60 million parameters • Techniques used for boosting up performance • ReLU nonlinearity • Training on Multiple GPUs • Overlapping max pooling • Data Augmentation • Dropout
• 6.
  Introduction > Methods> Results > Conclusion6 Overall Architecture • Trained with stochastic gradient descent on two NVIDIA GPUs for about a week (5~6 days) • 650,000 neurons, 60 million parameters, 630 million connections • The last layer contains 1,000 neurons which produces a distribution over the 1,000 class labels.
• 7.
  Introduction > Methods> Results > Conclusion7 Dataset • ImageNet § Over 15 million high-quality labeled images § About 22,000 categories § Collected from the web, labeled by humans on Amazon's Mechanical Turk § Variable-resolution images • ILSVRC Competition § ImageNet Large-Scale Pascal Visual Object Challenge § Annual competition of image classification at large scale § Subset of ImageNet § 1,000 categories with about 1,000 images each § 1.2M images in 1K categories § Classification: make 5 guesses about the image label
• 8.
  Introduction > Methods> Results > Conclusion8 Rectified Linear Units 𝑥 = 𝑤$ 𝑓 𝑍$ + 𝑤( 𝑓 𝑍( +𝑤) 𝑓 𝑍) x is called the total input to the neuron, and f(x) is its output Very bad (slow to train ) Very good (quick to train) f(x) = max(0,x)f(x) = tanh(x)
• 9.
  Introduction > Methods> Results > Conclusion9 Rectified Linear Units • Biological plausibility: One-sided, compared to the antisymmetry of tanh. • Sparse activation: For example, in a randomly initialized network, only about 50% of hidden units are activated (having a non-zero output). • Efficient gradient propagation: No vanishing gradient problem or exploding effect. • Efficient computation: Only comparison, addition and multiplication
• 10.
  Introduction > Methods> Results > Conclusion10 Training on Multiple GPUs • Spread across two GPUs • GTX 580 GPU with 3GB memory • Particularly well-suited to cross-GPU parallelization • Very efficient implementation of CNN on GPUs
• 11.
  Model Top-1 Top-5 Sparse coding[3] 47.1% 28.2% SIFT + FVs [4] 45.7% 25.7% CNN 37.5 17.0% Introduction > Methods > Results > Conclusion11 Results & Comparison •ILSVRC-2010 test set ILSVRC-2010 winner Previous best published result Our Method Comparison of results on ILSRVCs 2010 test set. In italics best results achieved by others.
• 12.
  Introduction > Methods> Results > Conclusion12 Conclusion • Large, deep convolutional neural networks for large scale image classification was proposed • 5 convolutional layers, 3 fully-connected layers • 650,000 neurons, 60 million parameters • Several techniques for boosting up performance • The proposed method won the ILSVRC-2012 • Achieved a winning top-5 error rate of 15.3%, compared to 26.2% achieved by the second-best entry
• 13.
  Introduction > Methods> Results > Conclusion13 Conclusion
• 14.
  Introduction > Methods> Results > Conclusion14 References [1] http://cs.nyu.edu/~fergus/tutorials/ deep_learning_cvpr12/fergus_dl_tutorial_final.pptx [2] reference : http://web.engr.illinois.edu/ ~slazebni/spring14/lec24_cnn.pdf [3] A. Berg, J. Deng, and L. Fei-Fei. Large scale visual recognition challenge 2010. www.imagenet.org/challenges. 2010. [4] S. Tara, Brian Kingsbury, A.-r. Mohamed and B. Ramabhadran, "Learning Filter Banks within a Deep [4] J.Sánchezand F.Perronnin.High-dimensional signature compression for large-scale image classification. In Computer Vision and Pattern Recognition(CVPR), 2011IEEEConferenceon,pages1665–1672.IEEE, 2011.
• 15.
  Introduction > Methods> Results > Conclusion15 Thank you for your attention Any Questions?
• 16.
  Introduction > Methods> Results > Conclusion16 Results 2012 • ILSVRC-2012 results Proposed method Top-5 error rate : 16.422% Runner-up Top-5 error rate : 26.172%
• 17.
  Introduction > Methods> Results > Conclusion17 Convolutional NNs
• 18.
  Introduction > Methods> Results > Conclusion18 Pooling • Spatial Pooling • Non-overlapping / overlapping regions • Sum or max Max Sum
• 19.
  Introduction > Methods> Results > Conclusion19 Dropout • Independently set each hidden unit activity to zero with 0.5 probability • Used in the two globally-connected hidden layers at the net's output