ExperienceThinking: Constrained Hyperparameter Optimization based on
  Knowledge and Pruning
  Machine learning algorithms are very sensitive to the hyperparameters, and
their evaluations are generally expensive. Users desperately need intelligent
methods to quickly optimize hyperparameter settings according to known
evaluation information, and thus reduce computational cost and promote
optimization efficiency. Motivated by this, we propose ExperienceThinking
algorithm to quickly find the best possible hyperparameter configuration of
machine learning algorithms within a few configuration evaluations.
ExperienceThinking design two novel methods, which intelligently infer optimal
configurations from two aspects: search space pruning and knowledge utilization
respectively. Two methods complement each other and solve the constrained
hyperparameter optimization problems effectively. To demonstrate the benefit of
ExperienceThinking, we compare it with 3 classical hyperparameter optimization
algorithms with a small number of configuration evaluations. The experimental
results present that our proposed algorithm provides superior results and
achieve better performance.
