title: ConvNet : Validation Loss not strongly decreasing but accuracy is improving
body: <p>Using <code>TensorFlow</code> I've build a simple <code>CNN</code> for classification.  It has the following definition:</p>
  
  <pre><code>Input Tensor : 32,32,1 Grayscale Image
  1 Conv Layer 3x3x32
  Relu Activated
  2x2 Max Pooled
  128 FC1
  43 FC2 # 43 classes
  </code></pre>
  
  <p>Full code can be found on this <a href="https://github.com/autojazari/gtsd-solution/blob/master/Solution-Cleaned.ipynb" rel="nofollow noreferrer">notebook on github</a></p>
  
  <p>The <code>validation loss</code> and <code>accuracy</code> at <code>Epochs</code> <strong>100</strong>, <strong>1000</strong>, <strong>2000</strong> are</p>
  
  <pre><code>epoch 100 validation loss 3.67, validation accuracy 12.05%
  epoch 1000 validation loss 3.234, validation accuracy 57.63%
  epoch 2750 validation loss 3.111, validation accuracy 69.25%
  </code></pre>
  
  <p>Unless I've misunderstood or have a bug somewhere, the network is learning.  However the validation loss is has only decreased very slightly.  </p>
  
  <p>What does that mean? How can I use this information to improve the network?</p>