title: Why use softmax as opposed to standard normalization?
body: <p>In the output layer of a neural network, it is typical to use the softmax function to approximate a probability distribution:</p>
  
  <p><img src="https://i.stack.imgur.com/r1MZm.png" alt="enter image description here"></p>
  
  <p>This is expensive to compute because of the exponents. Why not simply perform a Z transform so that all outputs are positive, and then normalise just by dividing all outputs by the sum of all outputs?</p>