title: Looking for suggestions on how to improve my CNN model - how much loss is too much?
body: <p>I've played around with the setup of my architecture a lot, the number of layers, pooling size, dropouts, etc but I always end up in the same ballpark: ~96-98 accuracy and loss between 2-6%. </p>
  
  <p>I'm training from a dataset of 78000 images, 26 classes (letters of the alphabet), and 3000 images per photo. I am satisfied with the accuracy, but are there any suggestions to reduce loss? Also How much loss do you think is too much? Anything else I should add to the model to make it more robust?  Code for the model is shown below: </p>
  
  <pre><code>model = Sequential()
  
  model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(200, 200, 1))) 
  model.add(MaxPooling2D((2, 2)))
  model.add(BatchNormalization())
  #model.add(Dropout(0.5))
  
  model.add(Conv2D(64, (3, 3), activation='relu', input_shape=(200, 200, 1))) 
  model.add(MaxPooling2D((2, 2)))
  model.add(BatchNormalization())
  #model.add(Dropout(0.5))
  
  model.add(Conv2D(64, (3, 3), activation='relu', input_shape=(200, 200, 1))) 
  model.add(MaxPooling2D((2, 2)))
  model.add(BatchNormalization())
  #model.add(Dropout(0.5))
  
  model.add(Flatten())
  model.add(Dense(128, activation='relu'))
  model.add(Dense(26, activation='softmax'))
  </code></pre>
  
  <p>Compile and Fit: </p>
  
  <pre><code>model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
  
  model.fit(X_train, y_train, epochs=10, batch_size=64, verbose=1, validation_data=(X_test, y_test))
  </code></pre>
  
  <p>Output: </p>
  
  <pre><code>Train on 54600 samples, validate on 23400 samples
  Epoch 1/10
  54600/54600 [==============================] - 68s 1ms/step - loss: 1.1217 - accuracy: 0.6622 - val_loss: 0.9795 - val_accuracy: 0.7394
  Epoch 2/10
  54600/54600 [==============================] - 67s 1ms/step - loss: 0.2377 - accuracy: 0.9219 - val_loss: 0.2300 - val_accuracy: 0.9277
  Epoch 3/10
  54600/54600 [==============================] - 67s 1ms/step - loss: 0.1184 - accuracy: 0.9627 - val_loss: 0.2746 - val_accuracy: 0.9286
  Epoch 4/10
  54600/54600 [==============================] - 67s 1ms/step - loss: 0.0755 - accuracy: 0.9761 - val_loss: 0.1850 - val_accuracy: 0.9517
  Epoch 5/10
  54600/54600 [==============================] - 69s 1ms/step - loss: 0.0669 - accuracy: 0.9801 - val_loss: 0.2044 - val_accuracy: 0.9450
  Epoch 6/10
  54600/54600 [==============================] - 69s 1ms/step - loss: 0.0520 - accuracy: 0.9848 - val_loss: 0.2265 - val_accuracy: 0.9485
  Epoch 7/10
  54600/54600 [==============================] - 72s 1ms/step - loss: 0.0481 - accuracy: 0.9865 - val_loss: 0.1709 - val_accuracy: 0.9559
  Epoch 8/10
  54600/54600 [==============================] - 66s 1ms/step - loss: 0.0370 - accuracy: 0.9905 - val_loss: 0.1534 - val_accuracy: 0.9659
  Epoch 9/10
  54600/54600 [==============================] - 66s 1ms/step - loss: 0.0335 - accuracy: 0.9912 - val_loss: 0.1181 - val_accuracy: 0.9703
  Epoch 10/10
  54600/54600 [==============================] - 66s 1ms/step - loss: 0.0277 - accuracy: 0.9921 - val_loss: 0.1204 - val_accuracy: 0.9704
  </code></pre>