title: What does b' mean when reconstructing an autoencdoer?
body: <p>Im just learning AutoEncoder, and I was reading from this website: <a href="http://deeplearning.net/tutorial/dA.html" rel="nofollow noreferrer">http://deeplearning.net/tutorial/dA.html</a> 
  and I didn't get what b' means. </p>
  
  <p>So I tried to build autoencoder with tied weights. </p>
  
  <p>For example, </p>
  
  <p>Encoding : hid = s(x*w+b)</p>
  
  <p>x : ( 1000, 2000 ) </p>
  
  <p>w : ( 2000, 500 )  </p>
  
  <p>b : ( 500, 1 )</p>
  
  <p>-> hid : ( 1000 , 500 )</p>
  
  <p>And when I decode encoded data, </p>
  
  <p>decode : y=s(hid*w'+b')</p>
  
  <p>w' : (500, 2000)</p>
  
  <p>so hid*w' will be (1000,2000) 
  and I have to add b', and its shape should be (2000,1) but I only have b with shape of ( 500, 1). </p>
  
  <p>What have I done wrong here? </p>
  
  <p>I found some code, and they just randomly made bais with shape of number of inputs, i.e., (2000,1) and optimized it. </p>