A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep
  Neural Networks
  We present a survey of ways in which existing scientific knowledge are
included when constructing models with neural networks. The inclusion of
domain-knowledge is of special interest not just to constructing scientific
assistants, but also, many other areas that involve understanding data using
human-machine collaboration. In many such instances, machine-based model
construction may benefit significantly from being provided with human-knowledge
of the domain encoded in a sufficiently precise form. This paper examines the
inclusion of domain-knowledge by means of changes to: the input, the
loss-function, and the architecture of deep networks. The categorisation is for
ease of exposition: in practice we expect a combination of such changes will be
employed. In each category, we describe techniques that have been shown to
yield significant changes in the performance of deep neural networks.
