Reducing The Search Space For Hyperparameter Optimization Using Group
  Sparsity
  We propose a new algorithm for hyperparameter selection in machine learning
algorithms. The algorithm is a novel modification of Harmonica, a spectral
hyperparameter selection approach using sparse recovery methods. In particular,
we show that a special encoding of hyperparameter space enables a natural
group-sparse recovery formulation, which when coupled with HyperBand (a
multi-armed bandit strategy) leads to improvement over existing hyperparameter
optimization methods such as Successive Halving and Random Search. Experimental
results on image datasets such as CIFAR-10 confirm the benefits of our
approach.
