Towards learning-to-learn
  In good old-fashioned artificial intelligence (GOFAI), humans specified
systems that solved problems. Much of the recent progress in AI has come from
replacing human insights by learning. However, learning itself is still usually
built by humans -- specifically the choice that parameter updates should follow
the gradient of a cost function. Yet, in analogy with GOFAI, there is no reason
to believe that humans are particularly good at defining such learning systems:
we may expect learning itself to be better if we learn it. Recent research in
machine learning has started to realize the benefits of that strategy. We
should thus expect this to be relevant for neuroscience: how could the correct
learning rules be acquired? Indeed, cognitive science has long shown that
humans learn-to-learn, which is potentially responsible for their impressive
learning abilities. Here we discuss ideas across machine learning,
neuroscience, and cognitive science that matter for the principle of
learning-to-learn.
