Bitwise Neural Networks
  Based on the assumption that there exists a neural network that efficiently
represents a set of Boolean functions between all binary inputs and outputs, we
propose a process for developing and deploying neural networks whose weight
parameters, bias terms, input, and intermediate hidden layer output signals,
are all binary-valued, and require only basic bit logic for the feedforward
pass. The proposed Bitwise Neural Network (BNN) is especially suitable for
resource-constrained environments, since it replaces either floating or
fixed-point arithmetic with significantly more efficient bitwise operations.
Hence, the BNN requires for less spatial complexity, less memory bandwidth, and
less power consumption in hardware. In order to design such networks, we
propose to add a few training schemes, such as weight compression and noisy
backpropagation, which result in a bitwise network that performs almost as well
as its corresponding real-valued network. We test the proposed network on the
MNIST dataset, represented using binary features, and show that BNNs result in
competitive performance while offering dramatic computational savings.
