Training Quantized Neural Networks to Global Optimality via Semidefinite
  Programming
  Neural networks (NNs) have been extremely successful across many tasks in
machine learning. Quantization of NN weights has become an important topic due
to its impact on their energy efficiency, inference time and deployment on
hardware. Although post-training quantization is well-studied, training optimal
quantized NNs involves combinatorial non-convex optimization problems which
appear intractable. In this work, we introduce a convex optimization strategy
to train quantized NNs with polynomial activations. Our method leverages hidden
convexity in two-layer neural networks from the recent literature, semidefinite
lifting, and Grothendieck's identity. Surprisingly, we show that certain
quantized NN problems can be solved to global optimality in polynomial-time in
all relevant parameters via semidefinite relaxations. We present numerical
examples to illustrate the effectiveness of our method.
