A Local Optima Network Analysis of the Feedforward Neural Architecture Space
This study investigates the use of local optima network (LON) analysis, a
derivative of the fitness landscape of candidate solutions, to characterise and
visualise the neural architecture space. The search space of feedforward neural
network architectures with up to three layers, each with up to 10 neurons, is
fully enumerated by evaluating trained model performance on a selection of data
sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple
global structures, with single global funnels in all cases but one. These
results yield early indication that LONs may provide a viable paradigm by which
to analyse and optimise neural architectures.