title: Implement Relu derivative in python numpy
body: <p>I'm trying to implement a function that computes the Relu derivative for each element in a matrix, and then return the result in a matrix. I'm using Python and Numpy.</p>
  
  <p>Based on other Cross Validation posts, the Relu derivative for x is
  1 when x > 0, 0 when x &lt; 0, undefined or 0 when x == 0</p>
  
  <p>Currently, I have the following code so far:</p>
  
  <pre><code>def reluDerivative(self, x):
      return np.array([self.reluDerivativeSingleElement(xi) for xi in x])
  
  def reluDerivativeSingleElement(self, xi):
      if xi &gt; 0:
          return 1
      elif xi &lt;= 0:
          return 0
  </code></pre>
  
  <p>Unfortunately, xi is an array because x is an matrix. reluDerivativeSingleElement function doesn't work on array. So I'm wondering is there a way to map values in a matrix to another matrix using numpy, like the exp function in numpy?</p>
  
  <p>Thanks a lot in advance.</p>