Deep Neural Network Hyperparameter Optimization with Orthogonal Array
  Tuning
  Deep learning algorithms have achieved excellent performance lately in a wide
range of fields (e.g., computer version). However, a severe challenge faced by
deep learning is the high dependency on hyper-parameters. The algorithm results
may fluctuate dramatically under the different configuration of
hyper-parameters. Addressing the above issue, this paper presents an efficient
Orthogonal Array Tuning Method (OATM) for deep learning hyper-parameter tuning.
We describe the OATM approach in five detailed steps and elaborate on it using
two widely used deep neural network structures (Recurrent Neural Networks and
Convolutional Neural Networks). The proposed method is compared to the
state-of-the-art hyper-parameter tuning methods including manually (e.g., grid
search and random search) and automatically (e.g., Bayesian Optimization) ones.
The experiment results state that OATM can significantly save the tuning time
compared to the state-of-the-art methods while preserving the satisfying
performance. The codes are open in GitHub
(https://github.com/xiangzhang1015/OATM)
