NeuroDiff: Scalable Differential Verification of Neural Networks using
  Fine-Grained Approximation
  As neural networks make their way into safety-critical systems, where
misbehavior can lead to catastrophes, there is a growing interest in certifying
the equivalence of two structurally similar neural networks. For example,
compression techniques are often used in practice for deploying trained neural
networks on computationally- and energy-constrained devices, which raises the
question of how faithfully the compressed network mimics the original network.
Unfortunately, existing methods either focus on verifying a single network or
rely on loose approximations to prove the equivalence of two networks. Due to
overly conservative approximation, differential verification lacks scalability
in terms of both accuracy and computational cost. To overcome these problems,
we propose NeuroDiff, a symbolic and fine-grained approximation technique that
drastically increases the accuracy of differential verification while achieving
many orders-of-magnitude speedup. NeuroDiff has two key contributions. The
first one is new convex approximations that more accurately bound the
difference neurons of two networks under all possible inputs. The second one is
judicious use of symbolic variables to represent neurons whose difference
bounds have accumulated significant error. We also find that these two
techniques are complementary, i.e., when combined, the benefit is greater than
the sum of their individual benefits. We have evaluated NeuroDiff on a variety
of differential verification tasks. Our results show that NeuroDiff is up to
1000X faster and 5X more accurate than the state-of-the-art tool.
