Continuous Ant-Based Neural Topology Search
  This work introduces a novel, nature-inspired neural architecture search
(NAS) algorithm based on ant colony optimization, Continuous Ant-based Neural
Topology Search (CANTS), which utilizes synthetic ants that move over a
continuous search space based on the density and distribution of pheromones, is
strongly inspired by how ants move in the real world. The paths taken by the
ant agents through the search space are utilized to construct artificial neural
networks (ANNs). This continuous search space allows CANTS to automate the
design of ANNs of any size, removing a key limitation inherent to many current
NAS algorithms that must operate within structures with a size predetermined by
the user. CANTS employs a distributed asynchronous strategy which allows it to
scale to large-scale high performance computing resources, works with a variety
of recurrent memory cell structures, and makes use of a communal weight sharing
strategy to reduce training time. The proposed procedure is evaluated on three
real-world, time series prediction problems in the field of power systems and
compared to two state-of-the-art algorithms. Results show that CANTS is able to
provide improved or competitive results on all of these problems, while also
being easier to use, requiring half the number of user-specified
hyper-parameters.
