title: Imbalanced Dataset Using Keras
body: <p>I am building a classifying ANN with python and the Keras library. I am using training the NN on an imbalanced dataset with 3 different classes. Class 1 is about 7.5 times as prevalent as Classes 2 and 3. As remedy, I took the advice of <a href="https://datascience.stackexchange.com/questions/13490/how-to-set-class-weights-for-imbalanced-classes-in-keras">this stackoverflow answer</a> and set my class weights as such:</p>
  
  <pre><code>class_weight = {0 : 1,
                  1 : 6.5,
                  2: 7.5}
  </code></pre>
  
  <p><strong>However, here is the problem:</strong> The ANN is predicting the 3 classes at equal rates! </p>
  
  <p>This is not useful because the dataset <em>is</em> imbalanced, and predicting the outcomes as each having a 33% chance is inaccurate. </p>
  
  <p><strong>Here is the question:</strong> How do I deal with an imbalanced dataset so that the ANN does not predict Class 1 every time, but also so that the ANN does not predict the classes with equal probability?</p>
  
  <p>Here is my code I am working with:</p>
  
  <pre><code>class_weight = {0 : 1,
  1 : 6.5,
  2: 7.5}
  
  # Making the ANN
  import keras
  from keras.models import Sequential
  from keras.layers import Dense
  from keras.layers import Dropout
  
  classifier = Sequential()
  
  
  # Adding the input layer and the first hidden layer with dropout
  classifier.add(Dense(activation = 'relu',
                       input_dim = 5,
                       units = 3,
                       kernel_initializer = 'uniform'))
  #Randomly drops 0.1, 10% of the neurons in the layer.
  classifier.add(Dropout(rate= 0.1))
  
  #Adding the second hidden layer
  classifier.add(Dense(activation = 'relu',
                       units = 3,
                       kernel_initializer = 'uniform'))
  #Randomly drops 0.1, 10% of the neurons in the layer.
  classifier.add(Dropout(rate = 0.1)) 
  
  # Adding the output layer
  classifier.add(Dense(activation = 'sigmoid',
                       units = 2,
                       kernel_initializer = 'uniform'))
  
  # Compiling the ANN
  classifier.compile(optimizer = 'adam',
                     loss = 'binary_crossentropy',
                     metrics = ['accuracy'])
  
  # Fitting the ANN to the training set
  classifier.fit(X_train, y_train, batch_size = 100, epochs = 100, class_weight = class_weight)
  </code></pre>