• TensorFlow Basic Concepts

    Hello Codacus

    To test that we have successfully installed tensorflow lets run this code to see if its working perfectly
    import tensorflow as tf
hello = tf.constant('Hello, Codacus!')
sess = tf.Session()
print(sess.run(hello))
    Python
    If there is no error, you should get “Hello, Codacus” in the output

    TensorFlow Basic Concepts

    There are the main concept in tensorflow that we need to understand.
    • placeholders
    • variables
    • operations
    • session
    these are the three things we will be dealing with when working with tensorflow. lets discuss about these one by one individually

    Placeholders

    Placeholders are the terminals/data point through which we will be feeding data into the network we will build. Its like gate point for our input and output data.
    lets create some placeholders
    inputs=tf.placeholder('float',[None,2],name='Input')
targets=tf.placeholder('float',name='Target')
    Python
    So we created two placeholders the data type that will feed to the placeholder is float , 2nd parameter is the shape of input. In tensorflow data flows through the network in form of matrix.
    so lets say we have 3 input neurons.
    then our input data should be a list of 3 values for example [1,2,3]
    Now lets say we have 4 sets of inputs for example

    import tensorflow as tfInput1 = tf.placeholder('float',name="Input_1")Input2 = tf.placeholder('float',name="Input_2")x=tf.Variable(0,dtype='float')output= tf.Variable(0,dtype='float')
    x= Input1*Input2output = x*x
    sess= tf.Session()print(sess.run(output, feed_dict={Input1:3,Input2:4}))

    [1,2,3]
[2,4,3]
[2,8,6]
[5,4,3]
    Python
    and we want to feed all the three inputs in a batch the shape on the input will be [4,3]
    Now lets say we don’t know how many input set we are going to feed at the same time. it can be 1 it can be 100 sets at a time. So we specify that with None.
    if we set the input shape of the placeholder as [None,3] then it will be able to take any sets of data in a single shot. that is what we did in out earlier code.
    In the second case I didn’t select any shape, in this case it will accept any shape. but there is a chance of getting error in runtime if the data the network is expecting has a different shape than the data we provided in the placeholder.
    The third parameter I used is name  it is not so important for the network to work. but it will be useful to give a name to every node of the network. This will help up to identify the nodes when we will be checking the network diagram in Tensor board 

    Variables


    Lets create some Variables
    weight1=tf.Variable(tf.random_normal(shape=[2,3],stddev=0.02),name="Weight1")
biases1=tf.Variable(tf.random_normal(shape=[3],stddev=0.02),name="Biases1")
    Python
    So we have two variables here. the first parameter in here is the values we want it initialize it with. in this case we initialized the first one as 2×3 matrix with random values with variation of +/- 0.02. for the shape the first one is the number of input connection that the layer will receive. and the 2nd one is the number of output connection that the layer will produce for the next layer.
    here is an example
    Neural Network
    For the bias I only selected 3 as shape because for the first bias neurons there is no input and it has 3 output connections.

    Operations

    Operations are the part where data get transferred from one layer to the next layer. using the weight variables we create which is basically the connection weights between neurons of two consecutive layers
    In our previous post we calculated the output of each neuron by summing the weighed output of all the previous layer neuron. we were doing that for each neuron in the layer using a for loop. But there is a better way of doing that. for example we have 2 input neuron and 3 hidden neuron. So there will be total 2×3 number of connections in between. we are presenting them in a 2d matrix
    number of connections in between. we are presenting them in a 2d matrix

    If we do matrix multiplication we will get  the below results
    so these are the same calculation we did in our previous post for each output using a loop to calculate the sum. Now to add the bias we can directly sum the bias with the results
    this is the final form of the output with bias.. okay let’s do this in tensorflow. To do this we need to do operations that is matrix multiplications
    hLayer=tf.matmul(inputs,weight1)
hLayer=hLayer+biases1
    Python
    So here is two matrix operations first one is matrix multiplications of inputs and the weights1 matrix which will produce the output without bias. and in the next line we did matrix addition which basically performed an element by element additions.
    Now this is what we call operations in tensorflow. It’s actually flowing the data from one layer.
    one last thing is still pending that is applying the activation functions in the layer outputs. so lets apply that
    hLayer=tf.sigmoid(hLayer, name='hActivation')
    Python
    Okay so we completed till hidden layer lets create the output layer also and finish the network
    weight2=tf.Variable(tf.random_normal(shape=[3,1],stddev=0.02),name="Weight2")
biases2=tf.Variable(tf.random_normal(shape=[1],stddev=0.02),name="Biases2")
    Python
    As we can see the output layer has only 1 neuron and the previous hidden layer had 3 neurons. So the weight matrix has 3 input and 1 output thus the shape is [3,1]. and the bias has only one neuron to connect to thus bias shape in [1]
    lets create the output layer
    output=tf.matmul(hLayer,weight2)
output=output+biases2
output=tf.sigmoid(output, name='outActivation')
    Python
    with this our network is complete.
    Optimization
    We still have’t done anything to calculate the error or any optimization methods to train the network
    #cost=tf.nn.softmax_cross_entropy_with_logits(logits=output,labels=targets)
# update  tf.nn.softmax_cross_entropy_with_logits is for classification problems only
# we will be using tf.squared_difference()
cost=tf.squared_difference(targets, output)
cost=tf.reduce_mean(cost)
optimizer=tf.train.AdamOptimizer().minimize(cost)
    Python
    The method softmax_cross_entropy_with_logits will take the output of the neuron and target placeholder where we will put out expected outputs. then it will calculate the error for each and every neuron in the layer.
    Update: I changes the cost function to tf.squared_difference()  which is similar to the same squared difference, and best suitable for this problem which is not a classification problem,  tf.squared_difference  takes two input fitsr is the target value and second is the predicted value.
    next we took that error in a variable cost and put operated reduce_mean  function on it and stored it in cost variable again. the reduce_mean  is taking that array of errors and calculating the average of them and returning a single value for the error.
    finally we are creating the optimizer to called AdamOptimizer to minimize the error cost . It’s basically do the same back propagation using the cost as starting error and propagate that error backwards while fixing the weights a little bit every time. Almost similar that we did in our own neural network in the last post.

    Session

    Till now what we did is, we described tensorflow what is the structure of the network and how data will flow. but the network is still not running and no operation has been done yet. So to run the network we need to start a tensorflow session.  any actual calculation will happen inside a session what ever tensorflow operation we perform will actually happen after we start the session and run it.
    So before starting the session lets create the data set first. I am going to make it work as another binary gate XOR gate
    of course we can do much complicated stuff but for the sake of simplicity we are going to make it work like an XOR gate. So lets generate the input outputs for this
    #generating inputs
import numpy as np

inp=[[0,0],[0,1],[1,0],[1,1]]
out=[[0],[1],[1],[0]]

inp=np.array(inp)
out=np.array(out)
    Python
    So before that we need our inputs. and we need it in numpy array format so we converted it to numpy array.
    Now lets start the session
    epochs=4000 # number of time we want to repeat

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    for i in range(epochs):
        error,_ =sess.run([cost,optimizer],feed_dict={inputs: inp,targets:out})
        print(i,error)
    Python
    So here we started a session and named it sess . After we setup the session the first thing we need to do is we need to initialize all the tensorflow variables that we created using  tf.global_variables_initializer().run() . 
    Now we started a for loop to repeat our training. now inside the loop we are using sess.run() . The first parameter in this is a list of operations that we want to do. In this case I want to know the error for each training and I also want to Optimize the network. So I put cost and the optimizer variable in the list.
    Te second parameter is the feed_dict . In this argument we will tell tensorflow from where it will take the data for all its placeholders. So we have 2 placeholders in our network inputs and targets we have to tell tensorflow from where it should take its data. We put that information in the feed_dict parameter.
    for inputs we set inp . and for targets we set out . Therefore we set feed_dict ={inputs:inp , targets:out}
    and now the output of sess.run() is the output of the individual operation that we put inside the list. that is the cost and the optimizer. the cost returns the average error and the optimizer doesn’t return any value so we set it as _ to ignore it .
    and finally we are printing those values.

    Lets Test the model

    We have trained the model now lets test its performance
    with tf.Session() as sess:
    tf.global_variables_initializer().run()
    for i in range(epochs):
        error,_ =sess.run([cost,optimizer],feed_dict={inputs: inp,targets:out})
        print(i,error)
    while True:
        a = input("type 1st input :")
        b = input("type 2nd input :")
        inp=[[a,b]]
        inp=np.array(inp)
        prediction=sess.run([output],feed_dict={inputs: inp})
        print(prediction)
    Python

    So here we go. after training we can test the network by running the output node and supplying data to the inputs placeholder  and finally printing it

    Saving The Session for later

    One thing to remember is all the variable that were tuned in the session is only remain in that state till the session is active. one we close the session the state of the network is lost. To avoid this we need the save the session. and reload it every time we want to use it again.
    to save the session we can use tf.train.Saver() to create a saver object
    saver = tf.train.Saver()
with tf.Session() as sess:
    tf.global_variables_initializer().run()
    for i in range(epochs):
        error,_ =sess.run([cost,optimizer],feed_dict={inputs: inp,targets:out})
        print(i,error)
    saver.save(sess, "model.ckpt") # saving the session with file name "model.ckpt"
    Python
    At the end of the for loop after the training we are saving the session with a file name “model.ckpt”. so now we can load it for later use.

    Restoring/Loading the session

    Once we have a saved session we can now restore it easily by calling the restore method of Saver class object
    saver = tf.train.Saver()
with tf.Session() as sess:
    saver.restore(sess, "model.ckpt")
    while True:
        a = input("type 1st input :")
        b = input("type 2nd input :")
        inp=[[a,b]]
        inp=np.array(inp)
        prediction=sess.run([output],feed_dict={inputs: inp})
        print(prediction)
    Python
    So This is how we can restore a previously trained model or later use. Thanks for reading I hope this post we help you to understand the basics of tensorflow.
    In the next post we will discuss how to use Tensorboard to visualize the network we design in a graphical representation and see the data flowing in real-time

    • 0 comments:

    Post a Comment

    New Research

    Attention Mechanism Based Multi Feature Fusion Forest for Hyperspectral Image Classification.

    CBS-GAN: A Band Selection Based Generative Adversarial Net for Hyperspectral Sample Generation.

    Multi-feature Fusion based Deep Forest for Hyperspectral Image Classification.

    New Research

    SAY HELLO TO ME

    ADDRESS

    388 Lumo Rd, Hongshan, Wuhan, Hubei, China

    EMAIL

    contact-m.zamanb@yahoo.com
    mostofa.zaman@cug.edu.cn

    TELEPHONE

    #
    #

    MOBILE

    +8615527370302,
    +8807171546477

    Copyright ©2019

    Copyright © 2017 Mohammad Mostofa Zaman. Designed By Portfolio Blogger Template | Distributed By Gooyaabi Templates
    MOHAMMAD MOSTOFA ZAMAN