title: Forward function in tensorflow
body: <p>I'm learning about neural network and reading a piece of code to train the neural network with mnist data set.
  In the test part of it, it defines the output variable: <code>y = mnist_forward.forward(x, None)</code> in the test function: </p>
  
  <pre><code>def test(mnist):
      with tf.Graph().as_default() as g:
          x = tf.placeholder(tf.float32, [None, mnist_forward.INPUT_NODE])
          y_ = tf.placeholder(tf.float32, [None, mnist_forward.OUTPUT_NODE])
          y = mnist_forward.forward(x, None)
  
          saver = tf.train.Saver()
          correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
          accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
  
          while True:
              with tf.Session() as sess:
                  tf.initialize_all_variables().run()
                  ckpt = tf.train.get_checkpoint_state(mnist_backward.MODEL_SAVE_PATH)
                  if ckpt and ckpt.model_checkpoint_path:
                      saver.restore(sess, ckpt.model_checkpoint_path)
                      global_step = ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1]
                      accuracy_score = sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})
                      print("After %s train steps, test accuracy = %g" % (global_step, accuracy_score))
                  else:
                      print('No checkpoint file found')
                      return
              time.sleep(TEST_INTERVAL_SECS)
  </code></pre>
  
  <p>However, the function <code>mnist_forward.forward</code> is defined as:</p>
  
  <pre><code>def forward(x, regularizer):
      w1 = get_weight([INPUT_NODE, LAYER1_NODE], regularizer)
      b1 = get_bias([LAYER1_NODE])
      y1 = tf.nn.relu(tf.matmul(x, w1) + b1)
  
      w2 = get_weight([LAYER1_NODE, OUTPUT_NODE], regularizer)
      b2 = get_bias([OUTPUT_NODE])
      y = tf.matmul(y1, w2) + b2
      return y
  </code></pre>
  
  <p>The function <code>get_weight</code> and <code>get_bias</code> are used to generate the parameters randomly. It seems that each time the forward function is called(to predict), the parameters will be regenerated.<br><br>
  I just cannot understand why the function call to generate parameters should be written inside the forward function(rather than outside) and how it truly works in the prediction process of the model?</p>