The Next Big Thing(s) in Unsupervised Machine Learning: Five Lessons
  from Infant Learning
  After a surge in popularity of supervised Deep Learning, the desire to reduce
the dependence on curated, labelled data sets and to leverage the vast
quantities of unlabelled data available recently triggered renewed interest in
unsupervised learning algorithms. Despite a significantly improved performance
due to approaches such as the identification of disentangled latent
representations, contrastive learning, and clustering optimisations, the
performance of unsupervised machine learning still falls short of its
hypothesised potential. Machine learning has previously taken inspiration from
neuroscience and cognitive science with great success. However, this has mostly
been based on adult learners with access to labels and a vast amount of prior
knowledge. In order to push unsupervised machine learning forward, we argue
that developmental science of infant cognition might hold the key to unlocking
the next generation of unsupervised learning approaches. Conceptually, human
infant learning is the closest biological parallel to artificial unsupervised
learning, as infants too must learn useful representations from unlabelled
data. In contrast to machine learning, these new representations are learned
rapidly and from relatively few examples. Moreover, infants learn robust
representations that can be used flexibly and efficiently in a number of
different tasks and contexts. We identify five crucial factors enabling
infants' quality and speed of learning, assess the extent to which these have
already been exploited in machine learning, and propose how further adoption of
these factors can give rise to previously unseen performance levels in
unsupervised learning.
