Optimising Optimisers with Push GP
  This work uses Push GP to automatically design both local and
population-based optimisers for continuous-valued problems. The optimisers are
trained on a single function optimisation landscape, using random
transformations to discourage overfitting. They are then tested for generality
on larger versions of the same problem, and on other continuous-valued
problems. In most cases, the optimisers generalise well to the larger problems.
Surprisingly, some of them also generalise very well to previously unseen
problems, outperforming existing general purpose optimisers such as CMA-ES.
Analysis of the behaviour of the evolved optimisers indicates a range of
interesting optimisation strategies that are not found within conventional
optimisers, suggesting that this approach could be useful for discovering novel
and effective forms of optimisation in an automated manner.
