Hyper-parameter estimation method with particle swarm optimization
  Particle swarm optimization (PSO) method cannot be directly used in the
problem of hyper-parameter estimation since the mathematical formulation of the
mapping from hyper-parameters to loss function or generalization accuracy is
unclear. Bayesian optimization (BO) framework is capable of converting the
optimization of the hyper-parameters into the optimization of an acquisition
function. The acquisition function is non-convex and multi-peak. So the problem
can be better solved by the PSO. The proposed method in this paper uses the
particle swarm method to optimize the acquisition function in the BO framework
to get better hyper-parameters. The performances of proposed method in both of
the classification and regression models are evaluated and demonstrated. The
results on several benchmark problems are improved.
