Rethinking Default Values: a Low Cost and Efficient Strategy to Define
  Hyperparameters
  Machine Learning (ML) algorithms have been increasingly applied to problems
from several different areas. Despite their growing popularity, their
predictive performance is usually affected by the values assigned to their
hyperparameters (HPs). As consequence, researchers and practitioners face the
challenge of how to set these values. Many users have limited knowledge about
ML algorithms and the effect of their HP values and, therefore, do not take
advantage of suitable settings. They usually define the HP values by trial and
error, which is very subjective, not guaranteed to find good values and
dependent on the user experience. Tuning techniques search for HP values able
to maximize the predictive performance of induced models for a given dataset,
but have the drawback of a high computational cost. Thus, practitioners use
default values suggested by the algorithm developer or by tools implementing
the algorithm. Although default values usually result in models with acceptable
predictive performance, different implementations of the same algorithm can
suggest distinct default values. To maintain a balance between tuning and using
default values, we propose a strategy to generate new optimized default values.
Our approach is grounded on a small set of optimized values able to obtain
predictive performance values better than default settings provided by popular
tools. After performing a large experiment and a careful analysis of the
results, we concluded that our approach delivers better default values.
Besides, it leads to competitive solutions when compared to tuned values,
making it easier to use and having a lower cost. We also extracted simple rules
to guide practitioners in deciding whether to use our new methodology or a HP
tuning approach.
