Regularized Binary Network Training
  There is a significant performance gap between Binary Neural Networks (BNNs)
and floating point Deep Neural Networks (DNNs). We propose to improve the
binary training method, by introducing a new regularization function that
encourages training weights around binary values. In addition, we add trainable
scaling factors to our regularization functions. Additionally, an improved
approximation of the derivative of the sign activation function in the backward
computation. These modifications are based on linear operations that are easily
implementable into the binary training framework. Experimental results on
ImageNet shows our method outperforms the traditional BNN method and XNOR-net.
