Hyperparameters Optimization in Deep Convolutional Neural Network /
  Bayesian Approach with Gaussian Process Prior
  Convolutional Neural Network is known as ConvNet have been extensively used
in many complex machine learning tasks. However, hyperparameters optimization
is one of a crucial step in developing ConvNet architectures, since the
accuracy and performance are reliant on the hyperparameters. This multilayered
architecture parameterized by a set of hyperparameters such as the number of
convolutional layers, number of fully connected dense layers & neurons, the
probability of dropout implementation, learning rate. Hence the searching the
hyperparameter over the hyperparameter space are highly difficult to build such
complex hierarchical architecture. Many methods have been proposed over the
decade to explore the hyperparameter space and find the optimum set of
hyperparameter values. Reportedly, Gird search and Random search are said to be
inefficient and extremely expensive, due to a large number of hyperparameters
of the architecture. Hence, Sequential model-based Bayesian Optimization is a
promising alternative technique to address the extreme of the unknown cost
function. The recent study on Bayesian Optimization by Snoek in nine
convolutional network parameters is achieved the lowerest error report in the
CIFAR-10 benchmark. This article is intended to provide the overview of the
mathematical concept behind the Bayesian Optimization over a Gaussian prior.
