AutoLR: An Evolutionary Approach to Learning Rate Policies
  The choice of a proper learning rate is paramount for good Artificial Neural
Network training and performance. In the past, one had to rely on experience
and trial-and-error to find an adequate learning rate. Presently, a plethora of
state of the art automatic methods exist that make the search for a good
learning rate easier. While these techniques are effective and have yielded
good results over the years, they are general solutions. This means the
optimization of learning rate for specific network topologies remains largely
unexplored. This work presents AutoLR, a framework that evolves Learning Rate
Schedulers for a specific Neural Network Architecture using Structured
Grammatical Evolution. The system was used to evolve learning rate policies
that were compared with a commonly used baseline value for learning rate.
Results show that training performed using certain evolved policies is more
efficient than the established baseline and suggest that this approach is a
viable means of improving a neural network's performance.
