Pruning and Slicing Neural Networks using Formal Verification
  Deep neural networks (DNNs) play an increasingly important role in various
computer systems. In order to create these networks, engineers typically
specify a desired topology, and then use an automated training algorithm to
select the network's weights. While training algorithms have been studied
extensively and are well understood, the selection of topology remains a form
of art, and can often result in networks that are unnecessarily large - and
consequently are incompatible with end devices that have limited memory,
battery or computational power. Here, we propose to address this challenge by
harnessing recent advances in DNN verification. We present a framework and a
methodology for discovering redundancies in DNNs - i.e., for finding neurons
that are not needed, and can be removed in order to reduce the size of the DNN.
By using sound verification techniques, we can formally guarantee that our
simplified network is equivalent to the original, either completely, or up to a
prescribed tolerance. Further, we show how to combine our technique with
slicing, which results in a family of very small DNNs, which are together
equivalent to the original. Our approach can produce DNNs that are
significantly smaller than the original, rendering them suitable for deployment
on additional kinds of systems, and even more amenable to subsequent formal
verification. We provide a proof-of-concept implementation of our approach, and
use it to evaluate our techniques on several real-world DNNs.
