Towards understanding deep learning with the natural clustering prior
  The prior knowledge (a.k.a. priors) integrated into the design of a machine
learning system strongly influences its generalization abilities. In the
specific context of deep learning, some of these priors are poorly understood
as they implicitly emerge from the successful heuristics and tentative
approximations of biological brains involved in deep learning design. Through
the lens of supervised image classification problems, this thesis investigates
the implicit integration of a natural clustering prior composed of three
statements: (i) natural images exhibit a rich clustered structure, (ii) image
classes are composed of multiple clusters and (iii) each cluster contains
examples from a single class. The decomposition of classes into multiple
clusters implies that supervised deep learning systems could benefit from
unsupervised clustering to define appropriate decision boundaries. Hence, this
thesis attempts to identify implicit clustering abilities, mechanisms and
hyperparameters in deep learning systems and evaluate their relevance for
explaining the generalization abilities of these systems. We do so through an
extensive empirical study of the training dynamics as well as the neuron- and
layer-level representations of deep neural networks. The resulting collection
of experiments provides preliminary evidence for the relevance of the natural
clustering prior for understanding deep learning.
