Rethinking supervised learning: insights from biological learning and
  from calling it by its name
  The renaissance of artificial neural networks was catalysed by the success of
classification models, tagged by the community with the broader term supervised
learning. The extraordinary results gave rise to a hype loaded with ambitious
promises and overstatements. Soon the community realised that the success owed
much to the availability of thousands of labelled examples and supervised
learning went, for many, from glory to shame: Some criticised deep learning as
a whole and others proclaimed that the way forward had to be alternatives to
supervised learning: predictive, unsupervised, semi-supervised and, more
recently, self-supervised learning. However, all these seem brand names, rather
than actual categories of a theoretically grounded taxonomy. Moreover, the call
to banish supervised learning was motivated by the questionable claim that
humans learn with little or no supervision and are capable of robust
out-of-distribution generalisation. Here, we review insights about learning and
supervision in nature, revisit the notion that learning and generalisation are
not possible without supervision or inductive biases and argue that we will
make better progress if we just call it by its name.
