Lifelong Bayesian Optimization
  Automatic Machine Learning (Auto-ML) systems tackle the problem of automating
the design of prediction models or pipelines for data science. In this paper,
we present Lifelong Bayesian Optimization (LBO), an online, multitask Bayesian
optimization (BO) algorithm designed to solve the problem of model selection
for datasets arriving and evolving over time. To be suitable for "lifelong"
Bayesian Optimization, an algorithm needs to scale with the ever increasing
number of acquisitions and should be able to leverage past optimizations in
learning the current best model. We cast the problem of model selection as a
black-box function optimization problem. In LBO, we exploit the correlation
between functions by using components of previously learned functions to speed
up the learning process for newly arriving datasets. Experiments on real and
synthetic data show that LBO outperforms standard BO algorithms applied
repeatedly on the data.
