Download as PDF, PPTX
Uploaded byAnoop Thomas Mathew
Data Driven Code
The document is a guide on the basics of neural networks, focusing on parameter optimization, sparse coding, and foundational mathematics like linear algebra and calculus. It includes practical elements like building a simple neural network and applying the XOR truth table example. Additionally, it provides resources and links for further learning in the field of deep learning.
More Related Content
Data Driven Code
- 1. Hacker’s Guide toNeural Networks a. k. a. Data Driven Code – 101 Anoop Thomas Mathew @atmb4u PyCon Canada 2016
- 2. WHAT WE WILLNOT COVER • Recent developments in • United States of America! ;) • Deep Learning, CNN, RNN, DCGAN etc. • Frameworks like TensorFlow, Theano, Keras etc. • Advanced concepts of neural networks
- 3. WHAT WE WILLCOVER • Basic Concepts of • Parameter Optimization • Entropy (Sparse Coding) • Little bit of Mathematics • Linear Algebra – Matrix Multiplication • Differential Calculus – Sigmoid Function (2 equations only) • (Try to) Build a 2 layer neural network • Way Forward
- 4. PARAMETER OPTIMIZATION • InfiniteMonkey Theorem • Any problem is fundamentally a parameter optimization problem 𝑎𝑥3 + 𝑏𝑥2 + 𝑐𝑥 + 𝑑 = 0
- 5. 0/1 noise bit data bits 28= 256 Imagine: • divided into 4 spaces • each bit – a feature • generalize data bits • attenuates noise bit ENTROPY 0/1 0/1 • Sparse Coding
- 6. LITTLE LINEAR ALGEBRA 𝑤1𝑤2 𝑤3 𝑤4 × 𝑥1 𝑥2 = 𝑤1 𝑥1 + 𝑤2 𝑥2 𝑤3 𝑥1 + 𝑤4 𝑥2
- 7. LITTLE LINEAR ALGEBRA(matrix multiplication) 𝑤1 𝑤2 𝑤3 𝑤4 × 𝑥1 𝑥2 = 𝑤1 𝑥1 + 𝑤2 𝑥2 𝑤3 𝑥1 + 𝑤4 𝑥2
- 8. ACTIVATION FUNCTION (differentialcalculus part) σ(x) = / 1+e−x (Sigmoid Function) 0 ∂σ(x) 02 = σ(x) × (1−σ(x)) (Derivative of Sigmoid Function) Stochastic Gradient Decent
- 9. QUICK RECAP • ActivationFunction(combining inputs to a representation) • Layers (Weight Matrices) • Network (collection of weight matrices) • Dataset (inputs and expect outputs) • Features (patterns in dataset) • Sparse Encoding (roomfor error) • Backpropagation (learn from past mistakes) • Parameter Optimization (correct values into weight matrix)
- 10. LET’S MAKE ONENOW ! CODE + DATA = MAGIC (Teach machine XOR truth table)
- 11.
- 12.
- 14.
- 15.
- 16. WAY FORWARD Play withNeural networks in the browser http://playground.tensorflow.org/ Comprehensive list of resources available online on Deep Learning https://github.com/ChristosChristofidis/awesome-deep-learning Very active online Machine Learning Community https://www.kaggle.com/
- 17. REFERENCE LINKS /IMAGE COURTESY • https://medium.com/technology-invention-and-more/how-to-build-a-simple- neural-network-in-9-lines-of-python-code-cc8f23647ca1 • https://medium.com/technology-invention-and-more/how-to-build-a-multi- layered-neural-network-in-python-53ec3d1d326a#.dis92by0e • http://iamtrask.github.io/2015/07/12/basic-python-network/ • http://iamtrask.github.io/2015/07/27/python-network-part2/ • http://sebastianraschka.com/faq/docs/logisticregr-neuralnet.html
- 18. THANK YOU // atm@infiniteloop.in •Code - https://github.com/atmb4u/data-driven-code • Slides – https://slideshare.net/atmb4u/data-driven-code • Medium - https://medium.com/@atmb4u/data-driven-code-101-17003f7b88ec