AutoML: Exploration v.s. Exploitation
  Building a machine learning (ML) pipeline in an automated way is a crucial
and complex task as it is constrained with the available time budget and
resources. This encouraged the research community to introduce several
solutions to utilize the available time and resources. A lot of work is done to
suggest the most promising classifiers for a given dataset using sundry of
techniques including meta-learning based techniques. This gives the autoML
framework the chance to spend more time exploiting those classifiers and tuning
their hyper-parameters. In this paper, we empirically study the hypothesis of
improving the pipeline performance by exploiting the most promising classifiers
within the limited time budget. We also study the effect of increasing the time
budget over the pipeline performance. The empirical results across autoSKLearn,
TPOT and ATM, show that exploiting the most promising classifiers does not
achieve a statistically better performance than exploring the entire search
space. The same conclusion is also applied for long time budgets.
