A contrastive rule for meta-learning
  Meta-learning algorithms leverage regularities that are present on a set of
tasks to speed up and improve the performance of a subsidiary learning process.
Recent work on deep neural networks has shown that prior gradient-based
learning of meta-parameters can greatly improve the efficiency of subsequent
learning. Here, we present a biologically plausible meta-learning algorithm
based on equilibrium propagation. Instead of explicitly differentiating the
learning process, our contrastive meta-learning rule estimates meta-parameter
gradients by executing the subsidiary process more than once. This avoids
reversing the learning dynamics in time and computing second-order derivatives.
In spite of this, and unlike previous first-order methods, our rule recovers an
arbitrarily accurate meta-parameter update given enough compute. We establish
theoretical bounds on its performance and present experiments on a set of
standard benchmarks and neural network architectures.
