Towards a Principled Learning Rate Adaptation for Natural Evolution
  Strategies
  Natural Evolution Strategies (NES) is a promising framework for black-box
continuous optimization problems. NES optimizes the parameters of a probability
distribution based on the estimated natural gradient, and one of the key
parameters affecting the performance is the learning rate. We argue that from
the viewpoint of the natural gradient method, the learning rate should be
determined according to the estimation accuracy of the natural gradient. To do
so, we propose a new learning rate adaptation mechanism for NES. The proposed
mechanism makes it possible to set a high learning rate for problems that are
relatively easy to optimize, which results in speeding up the search. On the
other hand, in problems that are difficult to optimize (e.g., multimodal
functions), the proposed mechanism makes it possible to set a conservative
learning rate when the estimation accuracy of the natural gradient seems to be
low, which results in the robust and stable search. The experimental
evaluations on unimodal and multimodal functions demonstrate that the proposed
mechanism works properly depending on a search situation and is effective over
the existing method, i.e., using the fixed learning rate.
