Dynamic Regret Analysis for Online Meta-Learning
  The online meta-learning framework has arisen as a powerful tool for the
continual lifelong learning setting. The goal for an agent is to quickly learn
new tasks by drawing on prior experience, while it faces with tasks one after
another. This formulation involves two levels: outer level which learns
meta-learners and inner level which learns task-specific models, with only a
small amount of data from the current task. While existing methods provide
static regret analysis for the online meta-learning framework, we establish
performance in terms of dynamic regret which handles changing environments from
a global prospective. We also build off of a generalized version of the
adaptive gradient methods that covers both ADAM and ADAGRAD to learn
meta-learners in the outer level. We carry out our analyses in a stochastic
setting, and in expectation prove a logarithmic local dynamic regret which
depends explicitly on the total number of iterations T and parameters of the
learner. Apart from, we also indicate high probability bounds on the
convergence rates of proposed algorithm with appropriate selection of
parameters, which have not been argued before.
