Machine Learning - Introduction to Tensorflow

Machine Learning - Introduction to Tensorflow

• 1.
  Introduction to TensorFlow PortlandData Science Group Created by Andrew Ferlitsch Community Outreach June, 2017
• 2.
  What is it? AnOpen Source Machine Learning Library released by Google in 2015 • Built on top of Google’s Inception v3 • Google’s most advanced image recognition system • Convolution Neural Network (CNN) • Available as a Python (or C++) Library
• 3.
  Basics – Whatis a Vector? Vector = [ number, number, … ] • A vector is an array of numbers. • In machine learning, a vector holds the feature values (variables) for a sample. 24 16 5 120000Vector = Age Education Level Years of Experience Income
• 4.
  Basics – Whatis a Matrix? Matrix = [ n ][ m ] • A matrix is a 2-dimensional array of numbers. • In machine learning, a matrix holds the feature values [columns] for samples [rows] in a dataset. 24 16 5 120000 27 12 8 70000 28 18 2 135000 Matrix = Age Education Level Years of Experience Income
• 5.
  What is aTensor? Tensor = [ n1 ][ n2 ] … [ nx ] • A tensor is a high dimensional array. • A tensor consists of features, where each feature is a vector or multi-dimensional array (e.g., such as in embedding). A Sample 1 2 3 4 5 6 7 8 9 Features
• 6.
  Design & Run •TensorFlow uses a Design & Run methodology. Design (symbolic representation) Run (bind the data and execute) Construct training model symbolically. Once designed, bind (connect) the data and execute the model. Data Output
• 7.
  Symbolic Representation asGraph • Tensors are inputs to tensor operations, which output new tensors. Variable (tensor) Variable (tensor) Op Output (tensor) User constructs a symbolic representation (graph) of how tensors flow through the network.
• 8.
  A Simple TensorFlowGraph Variable (matrix) Variable (matrix) MatMul Output (matrix) Two matrices are defined as input. The two matrices are dot multiplied. The output is the dot product of the two matrices.
• 9.
  TensorFlow with Python- Constants import tensorflow as tf # import the Tensorflow library x1 = tf.constant( 1.0 ) # define some constants x2 = tf.constant( 2.0 ) By default, values are floating point numbers sess = tf.session() # connect session for execution sess.run( [x1, x2] ) # run the graph • Design the Graph • Run the Graph [ 1.0, 2.0 ]
• 10.
  TensorFlow with Python- Operations import tensorflow as tf # import the tensorflow library x1 = tf.constant( 1.0 ) # define some constants x2 = tf.constant( 2.0 ) x3 = tf.add( x1, x2 ) # add inputs x1 and x2 sess = tf.session() # connect session for execution sess.run( x3 ) # run the graph • Design the Graph • Run the Graph 3.0 When node x3 is executed, it takes the inputs x1 and x2 and applies the tensor add operation. Since these are scalar values, they are added together to output a new scalar value.
• 11.
  TensorFlow with Python- Placeholders import tensorflow as tf # import the Tensorflow library x1 = tf.placeholder( tf.float ) # define some placeholders x2 = tf.placeholder( tf.float ) x3 = tf.add( x1, x2 ) # add inputs x1 and x2 sess = tf.session() # connect session for execution sess.run( x3, {x1: 1.0, x2: 2.0} ) # run the graph • Design the Graph • Run the Graph 3.0 When node x3 is executed, it binds the values to the inputs x1 and x2 and applies the tensor add operation. Since these are scalar values, they are added together to output a new scalar value. The data is bound at run time (parameterized variables).
• 12.
  TensorFlow with Python- Binding import tensorflow as tf # import the Tensorflow library x1 = tf.placeholder( tf.float, shape=[2] ) # define some placeholders x2 = tf.placeholder( tf.float, shape=[2] ) x3 = tf.add( x1, x2 ) # add inputs x1 and x2 sess = tf.session() # connect session for execution sess.run( x3, {x1: [1.0,2.0], x2: [3.0,4.0]} ) # run the graph • Design the Graph • Run the Graph [ 4.0, 6.0 ] Bound an array instead of a scalar value Matrix addition
• 13.
  Multiple Operation TensorFlowGraph Variable (matrix) Variable (matrix) MatAdd MatMul The two matrices are added. The output is the dot product (weight) of the addition of the two matrices. Output (matrix) Constant (Scalar) Weight applied to operation.
• 14.
  TensorFlow with Python– Multi-Op import tensorflow as tf # import the Tensorflow library x1 = tf.placeholder( tf.float ) # define some placeholders x2 = tf.placeholder( tf.float ) x3 = tf.add( x1, x2 ) # add inputs x1 and x2 x4 = tf.muliply( x3, 3.0 ) # multiply the result by 3 sess = tf.session() # connect session for execution sess.run( x4, {x1: [1.0,2.0], x2: [3.0,4.0]} ) # run the graph • Design the Graph • Run the Graph [ 12.0, 18.0 ] Matrix multiply (weight) applied to Matrix addition
• 15.
  TensorFlow with Python– Operators import tensorflow as tf # import the Tensorflow library x1 = tf.placeholder( tf.float, shape=[2] ) # define some variables x2 = tf.placeholder( tf.float, shape=[2] ) x3 = x1 + x2 # add inputs x1 and x2 x4 = x3 * 3.0 # multiply the result by 3 sess = tf.session() # connect session for execution sess.run( x4, {x1: [1.0,2.0], x2: [3.0,4.0]} ) # run the graph • Design the Graph • Run the Graph [ 12.0, 18.0 ] Shortcut (or inline) operator
• 16.
  TensorFlow with Python– Variables import tensorflow as tf # import the Tensorflow library a = tf.Variable( tf.float ) # define some variables b = tf.Variable( tf.float ) x = tf.placeholder( tf.float, shape=[4] ) yhat = a + b * x # simple linear regression sess = tf.session() # connect session for execution init = tf_global_variables_initializer() # initialize global variables sess.run(init) sess.run( yhat, {x: [1,2,3,4]} ) # run the graph • Design the Graph • Run the Graph [ 0, 0.30000001, 0.60000001, 0.90000001 ] Variables must be explicitly initialized prior to running the graph Run simple linear regression for x = 1, 2, 3 and 4 Example from tensorflow.org
• 17.
  TensorFlow with Python– Loss Function y = tf.placeholder( tf.float ) # labels (actual values) squares = tf.square( yhat – y ) # square of error loss = tf.reduce_sum( squares ) # summation of squared errors sess = tf.session() # connect session for execution init = tf_global_variables_initializer() # initialize global variables sess.run(init) sess.run( loss, {x: [1,2,3,4], y:[0,-1,-2,-3]} ) # execute the graph • Design the Graph • Run the Graph 23.66 Sum up the squared difference of the actual and predicted values. Run simple linear regression for x = 1, 2, 3 and 4 with actual values 0, -1, -2 and -3. Example from tensorflow.org
• 18.
  TensorFlow with Python– tf.train optimizer = tf.train.GradientDescentOptimizer( 0.1 ) train = optimizer.minimize( loss ) # training model sess.run(init) for i in range(0,1000): # train over 1000 iterations sess.run( train, {x: [1,2,3,4], y:[0,-1,-2,-3]} ) # train the model sess.run([ b,w ] ) • Design the Graph • Run the Graph [0.99999082, -0.9999969] Learning rate Train the simple linear regression over 1000 iterations to minimize the loss function. Example from tensorflow.org
• 19.
  TensorFlow and tf.contrib.learn •Not Covered in this tutorial • Is a high-level wrapper built on top of Tensorflow • For Developers – hides low-level implementation tf.contrib.learn is a high-level TensorFlow library that simplifies the mechanics of machine learning, including the following: • running training loops • running evaluation loops • managing data sets • managing feeding From tensorflow.org