title: stopping gradient optimizer in TensorFlow
body: <p>I'm trying to build a simple neural network in Tensorflow, but I have a question about gradient optimization.</p>
  
  <p>It might be a naive question, but do I have to set conditions to <em>stop</em> the optimizer?  Below is a sample printout from my network and you can see that after iteration (batch gradient descent of all data) 66, the cost begins to increase again.  So is it up to <em>me</em> to make sure the optimizer stops at this point?  (note: I didn't put all the output here, but the cost begins to increase exponentially as the number of iterations increase).</p>
  
  <p>Thanks for any guidance.</p>
  
  <pre><code>iteration 64 with average cost of 654.621 and diff of 0.462708
  iteration 65 with average cost of 654.364 and diff of 0.257202
  iteration 66 with average cost of 654.36 and diff of 0.00384521 
  iteration 67 with average cost of 654.663 and diff of -0.302368
  iteration 68 with average cost of 655.328 and diff of -0.665161
  iteration 69 with average cost of 656.423 and diff of -1.09497
  iteration 70 with average cost of 658.011 and diff of -1.58826
  </code></pre>