Interpreting Deep Learning: The Machine Learning Rorschach Test?
  Theoretical understanding of deep learning is one of the most important tasks
facing the statistics and machine learning communities. While deep neural
networks (DNNs) originated as engineering methods and models of biological
networks in neuroscience and psychology, they have quickly become a centerpiece
of the machine learning toolbox. Unfortunately, DNN adoption powered by recent
successes combined with the open-source nature of the machine learning
community, has outpaced our theoretical understanding. We cannot reliably
identify when and why DNNs will make mistakes. In some applications like text
translation these mistakes may be comical and provide for fun fodder in
research talks, a single error can be very costly in tasks like medical
imaging. As we utilize DNNs in increasingly sensitive applications, a better
understanding of their properties is thus imperative. Recent advances in DNN
theory are numerous and include many different sources of intuition, such as
learning theory, sparse signal analysis, physics, chemistry, and psychology. An
interesting pattern begins to emerge in the breadth of possible
interpretations. The seemingly limitless approaches are mostly constrained by
the lens with which the mathematical operations are viewed. Ultimately, the
interpretation of DNNs appears to mimic a type of Rorschach test --- a
psychological test wherein subjects interpret a series of seemingly ambiguous
ink-blots. Validation for DNN theory requires a convergence of the literature.
We must distinguish between universal results that are invariant to the
analysis perspective and those that are specific to a particular network
configuration. Simultaneously we must deal with the fact that many standard
statistical tools for quantifying generalization or empirically assessing
important network features are difficult to apply to DNNs.
