A meta-learning recommender system for hyperparameter tuning: predicting
  when tuning improves SVM classifiers
  For many machine learning algorithms, predictive performance is critically
affected by the hyperparameter values used to train them. However, tuning these
hyperparameters can come at a high computational cost, especially on larger
datasets, while the tuned settings do not always significantly outperform the
default values. This paper proposes a recommender system based on meta-learning
to identify exactly when it is better to use default values and when to tune
hyperparameters for each new dataset. Besides, an in-depth analysis is
performed to understand what they take into account for their decisions,
providing useful insights. An extensive analysis of different categories of
meta-features, meta-learners, and setups across 156 datasets is performed.
Results show that it is possible to accurately predict when tuning will
significantly improve the performance of the induced models. The proposed
system reduces the time spent on optimization processes, without reducing the
predictive performance of the induced models (when compared with the ones
obtained using tuned hyperparameters). We also explain the decision-making
process of the meta-learners in terms of linear separability-based hypotheses.
Although this analysis is focused on the tuning of Support Vector Machines, it
can also be applied to other algorithms, as shown in experiments performed with
decision trees.
