How deep should be the depth of convolutional neural networks: a
  backyard dog case study
  The work concerns the problem of reducing a pre-trained deep neuronal network
to a smaller network, with just few layers, whilst retaining the network's
functionality on a given task
  The proposed approach is motivated by the observation that the aim to deliver
the highest accuracy possible in the broadest range of operational conditions,
which many deep neural networks models strive to achieve, may not necessarily
be always needed, desired, or even achievable due to the lack of data or
technical constraints. In relation to the face recognition problem, we
formulated an example of such a usecase, the `backyard dog' problem. The
`backyard dog', implemented by a lean network, should correctly identify
members from a limited group of individuals, a `family', and should distinguish
between them. At the same time, the network must produce an alarm to an image
of an individual who is not in a member of the family. To produce such a
network, we propose a shallowing algorithm. The algorithm takes an existing
deep learning model on its input and outputs a shallowed version of it. The
algorithm is non-iterative and is based on the Advanced Supervised Principal
Component Analysis. Performance of the algorithm is assessed in exhaustive
numerical experiments. In the above usecase, the `backyard dog' problem, the
method is capable of drastically reducing the depth of deep learning neural
networks, albeit at the cost of mild performance deterioration.
  We developed a simple non-iterative method for shallowing down pre-trained
deep networks. The method is generic in the sense that it applies to a broad
class of feed-forward networks, and is based on the Advanced Supervise
Principal Component Analysis. The method enables generation of families of
smaller-size shallower specialized networks tuned for specific operational
conditions and tasks from a single larger and more universal legacy network.
