title: Tensorflow: loss decreasing, but accuracy stable
body: <p>My team is training a CNN in Tensorflow for binary classification of damaged/acceptable parts. We created our code by modifying the cifar10 example code. In my prior experience with Neural Networks, I always trained until the loss was very close to 0 (well below 1). However, we are now evaluating our model with a validation set during training (on a separate GPU), and it seems like the precision stopped increasing after about 6.7k steps, while the loss is still dropping steadily after over 40k steps. Is this due to overfitting? Should we expect to see another spike in accuracy once the loss is very close to zero? The current max accuracy is not acceptable. Should we kill it and keep tuning? What do you recommend? Here is our modified code and graphs of the training process.</p>
  
  <p><a href="https://gist.github.com/justineyster/6226535a8ee3f567e759c2ff2ae3776b" rel="noreferrer">https://gist.github.com/justineyster/6226535a8ee3f567e759c2ff2ae3776b</a></p>
  
  <p><a href="https://imgur.com/a/zMzGb" rel="noreferrer">Precision and Loss Images</a></p>