Hierarchical Collaborative Hyper-parameter Tuning
Hyper-parameter Tuning is among the most critical stages in building machine
learning solutions. This paper demonstrates how multi-agent systems can be
utilized to develop a distributed technique for determining near-optimal values
for any arbitrary set of hyper-parameters in a machine learning model. The
proposed method employs a distributedly formed hierarchical agent-based
architecture for the cooperative searching procedure of tuning hyper-parameter
values. The presented generic model is used to develop a guided randomized
agent-based tuning technique, and its behavior is investigated in both machine
learning and global function optimization applications. According the empirical
results, the proposed model outperformed both of its underlying randomized
tuning strategies in terms of classification error and function evaluations,
notably in higher number of dimensions.