Modularity in NEAT Reinforcement Learning Networks
  Modularity is essential to many well-performing structured systems, as it is
a useful means of managing complexity [8]. An analysis of modularity in neural
networks produced by machine learning algorithms can offer valuable insight
into the workings of such algorithms and how modularity can be leveraged to
improve performance. However, this property is often overlooked in the
neuroevolutionary literature, so the modular nature of many learning algorithms
is unknown. This property was assessed on the popular algorithm "NeuroEvolution
of Augmenting Topologies" (NEAT) for standard simulation benchmark control
problems due to NEAT's ability to optimise network topology. This paper shows
that NEAT networks seem to rapidly increase in modularity over time with the
rate and convergence dependent on the problem. Interestingly, NEAT tends
towards increasingly modular networks even when network fitness converges. It
was shown that the ideal level of network modularity in the explored parameter
space is highly dependent on other network variables, dispelling theories that
modularity has a straightforward relationship to network performance. This is
further proven in this paper by demonstrating that rewarding modularity
directly did not improve fitness.
