title: The output of softmax makes the binary cross entropy's output NAN, what should I do?
body: <p>I have implemented a neural network in Tensorflow where the last layer is a convolution layer, I feed the output of this convolution layer into a softmax activation function then I feed it to a cross-entropy loss function which is defined as follows along with the labels but the problem is I got NAN as the output of my loss function and I figured out it is because I have 1 in the output of softmax. So, my question is what should I do in this case?
  My input is a 16 by 16 image where I have 0 and 1 as the values of each pixel (binary classification)</p>
  
  <p>My loss function:</p>
  
  <pre><code>#Loss function
  def loss(prediction, label):
      #with tf.variable_scope("Loss") as Loss_scope:
      log_pred = tf.log(prediction, name='Prediction_Log')
      log_pred_2 = tf.log(1-prediction, name='1-Prediction_Log')
      cross_entropy = -tf.multiply(label, log_pred) - tf.multiply((1-label), log_pred_2) 
  
      return cross_entropy
  </code></pre>