Meta-strategy for Learning Tuning Parameters with Guarantees
  Online learning methods, like the online gradient algorithm (OGA) and
exponentially weighted aggregation (EWA), often depend on tuning parameters
that are difficult to set in practice. We consider an online meta-learning
scenario, and we propose a meta-strategy to learn these parameters from past
tasks. Our strategy is based on the minimization of a regret bound. It allows
to learn the initialization and the step size in OGA with guarantees. It also
allows to learn the prior or the learning rate in EWA. We provide a regret
analysis of the strategy. It allows to identify settings where meta-learning
indeed improves on learning each task in isolation.
