Safe Crossover of Neural Networks Through Neuron Alignment
  One of the main and largely unexplored challenges in evolving the weights of
neural networks using genetic algorithms is to find a sensible crossover
operation between parent networks. Indeed, naive crossover leads to
functionally damaged offspring that do not retain information from the parents.
This is because neural networks are invariant to permutations of neurons,
giving rise to multiple ways of representing the same solution. This is often
referred to as the competing conventions problem. In this paper, we propose a
two-step safe crossover(SC) operator. First, the neurons of the parents are
functionally aligned by computing how well they correlate, and only then are
the parents recombined. We compare two ways of measuring relationships between
neurons: Pairwise Correlation (PwC) and Canonical Correlation Analysis (CCA).
We test our safe crossover operators (SC-PwC and SC-CCA) on MNIST and CIFAR-10
by performing arithmetic crossover on the weights of feed-forward neural
network pairs. We show that it effectively transmits information from parents
to offspring and significantly improves upon naive crossover. Our method is
computationally fast,can serve as a way to explore the fitness landscape more
efficiently and makes safe crossover a potentially promising operator in future
neuroevolution research and applications.
