title: why biases change more frequent then weights?
body: <p>I am visualizing biases and weights of fully connected part of my network.
  I see that biases change more frequent then weights and I am trying to understand what does it mean.
  In both cases I just took the weights and biases and presented them as histogram.</p>
  
  <p><a href="https://i.stack.imgur.com/ZtE9r.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZtE9r.png" alt="enter image description here"></a></p>
  
  <p>here we see:
   bias which is 1x256 parameters.
   weights which is matrix of 256X512 parameters.</p>
  
  <p>Can be the reason that weight hardly change in visualization due to high number of numbers that just fall at the same bins ? can be better way to visualize the weight learning? or there some other reason.</p>
  
  <p>Another part of the network that connects 16 neurons 1 neuron.
  <a href="https://i.stack.imgur.com/wEsgK.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wEsgK.png" alt="enter image description here"></a></p>