Compositional Processing Emerges in Neural Networks Solving Math
  Problems
  A longstanding question in cognitive science concerns the learning mechanisms
underlying compositionality in human cognition. Humans can infer the structured
relationships (e.g., grammatical rules) implicit in their sensory observations
(e.g., auditory speech), and use this knowledge to guide the composition of
simpler meanings into complex wholes. Recent progress in artificial neural
networks has shown that when large models are trained on enough linguistic
data, grammatical structure emerges in their representations. We extend this
work to the domain of mathematical reasoning, where it is possible to formulate
precise hypotheses about how meanings (e.g., the quantities corresponding to
numerals) should be composed according to structured rules (e.g., order of
operations). Our work shows that neural networks are not only able to infer
something about the structured relationships implicit in their training data,
but can also deploy this knowledge to guide the composition of individual
meanings into composite wholes.
