Weighted Random Search for CNN Hyperparameter Optimization
  Nearly all model algorithms used in machine learning use two different sets
of parameters: the training parameters and the meta-parameters
(hyperparameters). While the training parameters are learned during the
training phase, the values of the hyperparameters have to be specified before
learning starts. For a given dataset, we would like to find the optimal
combination of hyperparameter values, in a reasonable amount of time. This is a
challenging task because of its computational complexity. In previous work
[11], we introduced the Weighted Random Search (WRS) method, a combination of
Random Search (RS) and probabilistic greedy heuristic. In the current paper, we
compare the WRS method with several state-of-the art hyperparameter
optimization methods with respect to Convolutional Neural Network (CNN)
hyperparameter optimization. The criterion is the classification accuracy
achieved within the same number of tested combinations of hyperparameter
values. According to our experiments, the WRS algorithm outperforms the other
methods.
