Snowflake: Scaling GNNs to High-Dimensional Continuous Control via
  Parameter Freezing
  Recent research has shown that graph neural networks (GNNs) can learn
policies for locomotion control that are as effective as a typical multi-layer
perceptron (MLP), with superior transfer and multi-task performance (Wang et
al., 2018; Huang et al., 2020). Results have so far been limited to training on
small agents, with the performance of GNNs deteriorating rapidly as the number
of sensors and actuators grows. A key motivation for the use of GNNs in the
supervised learning setting is their applicability to large graphs, but this
benefit has not yet been realised for locomotion control. We identify the
weakness with a common GNN architecture that causes this poor scaling:
overfitting in the MLPs within the network that encode, decode, and propagate
messages. To combat this, we introduce Snowflake, a GNN training method for
high-dimensional continuous control that freezes parameters in parts of the
network that suffer from overfitting. Snowflake significantly boosts the
performance of GNNs for locomotion control on large agents, now matching the
performance of MLPs, and with superior transfer properties.
