Universal Approximation Theorems of Fully Connected Binarized Neural
  Networks
  Neural networks (NNs) are known for their high predictive accuracy in complex
learning problems. Beside practical advantages, NNs also indicate favourable
theoretical properties such as universal approximation (UA) theorems. Binarized
Neural Networks (BNNs) significantly reduce time and memory demands by
restricting the weight and activation domains to two values. Despite the
practical advantages, theoretical guarantees based on UA theorems of BNNs are
rather sparse in the literature. We close this gap by providing UA theorems for
fully connected BNNs under the following scenarios: (1) for binarized inputs,
UA can be constructively achieved under one hidden layer; (2) for inputs with
real numbers, UA can not be achieved under one hidden layer but can be
constructively achieved under two hidden layers for Lipschitz-continuous
functions. Our results indicate that fully connected BNNs can approximate
functions universally, under certain conditions.
