Extrapolatable Relational Reasoning With Comparators in Low-Dimensional
  Manifolds
  While modern deep neural architectures generalise well when test data is
sampled from the same distribution as training data, they fail badly for cases
when the test data distribution differs from the training distribution even
along a few dimensions. This lack of out-of-distribution generalisation is
increasingly manifested when the tasks become more abstract and complex, such
as in relational reasoning. In this paper we propose a neuroscience-inspired
inductive-biased module that can be readily amalgamated with current neural
network architectures to improve out-of-distribution (o.o.d) generalisation
performance on relational reasoning tasks. This module learns to project
high-dimensional object representations to low-dimensional manifolds for more
efficient and generalisable relational comparisons. We show that neural nets
with this inductive bias achieve considerably better o.o.d generalisation
performance for a range of relational reasoning tasks. We finally analyse the
proposed inductive bias module to understand the importance of lower dimension
projection, and propose an augmentation to the algorithmic alignment theory to
better measure algorithmic alignment with generalisation.
