NeuralDivergence: Exploring and Understanding Neural Networks by
  Comparing Activation Distributions
  As deep neural networks are increasingly used in solving high-stake problems,
there is a pressing need to understand their internal decision mechanisms.
Visualization has helped address this problem by assisting with interpreting
complex deep neural networks. However, current tools often support only single
data instances, or visualize layers in isolation. We present NeuralDivergence,
an interactive visualization system that uses activation distributions as a
high-level summary of what a model has learned. NeuralDivergence enables users
to interactively summarize and compare activation distributions across layers,
classes, and instances (e.g., pairs of adversarial attacked and benign images),
helping them gain better understanding of neural network models.
