title: Keras RNN LSTM accuracy not changing
body: <p><strong>X_train</strong></p>
  
  <pre><code>[[0 1 1 1 1 0 0 0 0 0]
   [0 1 1 1 1 0 0 0 0 0]
   [0 1 1 1 0 0 0 0 0 0]
   [0 1 0 0 0 1 1 0 0 0]
   [0 1 0 0 0 1 1 0 0 0]
   [0 1 1 1 1 0 0 0 0 0]]
  </code></pre>
  
  <p><strong>y_train</strong></p>
  
  <pre><code>1
  1
  1
  0
  0
  1
  </code></pre>
  
  <p>The third and 4th column in X_train are a clear indicator of the output. I am trying out RNN with LSTM so I have chosen this sample data and I want to overfit this.</p>
  
  <p>The accuracy is not changing at all even after <strong>50</strong> epochs of training - </p>
  
  <pre><code>Epoch 1/60
  6/6 [==============================] - 1s - loss: 5.3141 - acc: 0.6667
  Epoch 2/60
  6/6 [==============================] - 1s - loss: 5.3141 - acc: 0.6667
  Epoch 3/60
  6/6 [==============================] - 1s - loss: 5.3141 - acc: 0.6667
  Epoch 4/60
  6/6 [==============================] - 1s - loss: 5.3141 - acc: 0.6667
  Epoch 5/60
  6/6 [==============================] - 1s - loss: 5.3141 - acc: 0.6667
  </code></pre>
  
  <p><strong>Model</strong></p>
  
  <pre><code>model = Sequential()
  model.add(Embedding(MAX_NB_WORDS, embedding_vecor_length, input_length=max_length,batch_input_shape=( batch_size, input_dim)))
  model.add(LSTM(10, return_sequences=False))
  model.add(Dense(1, activation='softmax'))
  </code></pre>
  
  <p><strong>Parameters</strong></p>
  
  <pre><code>MAX_NB_WORDS = 10
  embedding_vecor_length = 32
  max_length = 10
  batch_size = 2
  input_dim = max_length
  </code></pre>
  
  <p>I am using Theano backend. Probably something missing very obvious. Please help!</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>Apologies for providing half baked stuff. I am compiling the model thus - </p>
  
  <pre><code>opt = SGD(lr=0.001)
  model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])
  print(model.summary())
  
  print np.shape(X_train)
  
  callbacks = [
              # EarlyStopping(monitor='val_loss', patience=3, verbose=2),
              RemoteMonitor(root='http://localhost:9000'),
              ModelCheckpoint(filepath="/tmp/weights.hdf5", monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto')
          ]
  
  print X_train
  print y_train
  
  history = model.fit(X_train, y_train, nb_epoch=60, batch_size=batch_size, callbacks=callbacks) #, validation_data=(X_val, y_val)
  </code></pre>
  
  <p><strong>UPDATE2</strong></p>
  
  <p>Rectified this by changing the activation function from 'softmax' to 'sigmoid'
  A proper explanation is missing. I guess the "why" should be the answer not the how.</p>