On the Origin of Species of Self-Supervised Learning
  In the quiet backwaters of cs.CV, cs.LG and stat.ML, a cornucopia of new
learning systems is emerging from a primordial soup of mathematics-learning
systems with no need for external supervision. To date, little thought has been
given to how these self-supervised learners have sprung into being or the
principles that govern their continuing diversification. After a period of
deliberate study and dispassionate judgement during which each author set their
Zoom virtual background to a separate Galapagos island, we now entertain no
doubt that each of these learning machines are lineal descendants of some older
and generally extinct species. We make five contributions: (1) We gather and
catalogue row-major arrays of machine learning specimens, each exhibiting
heritable discriminative features; (2) We document a mutation mechanism by
which almost imperceptible changes are introduced to the genotype of new
systems, but their phenotype (birdsong in the form of tweets and vestigial
plumage such as press releases) communicates dramatic changes; (3) We propose a
unifying theory of self-supervised machine evolution and compare to other
unifying theories on standard unifying theory benchmarks, where we establish a
new (and unifying) state of the art; (4) We discuss the importance of digital
biodiversity, in light of the endearingly optimistic Paris Agreement.
