A HINT from Arithmetic: On Systematic Generalization of Perception,
  Syntax, and Semantics
  Inspired by humans' remarkable ability to master arithmetic and generalize to
unseen problems, we present a new dataset, HINT, to study machines' capability
of learning generalizable concepts at three different levels: perception,
syntax, and semantics. In particular, concepts in HINT, including both digits
and operators, are required to learn in a weakly-supervised fashion: Only the
final results of handwriting expressions are provided as supervision. Learning
agents need to reckon how concepts are perceived from raw signals such as
images (i.e., perception), how multiple concepts are structurally combined to
form a valid expression (i.e., syntax), and how concepts are realized to afford
various reasoning tasks (i.e., semantics). With a focus on systematic
generalization, we carefully design a five-fold test set to evaluate both the
interpolation and the extrapolation of learned concepts. To tackle this
challenging problem, we propose a neural-symbolic system by integrating neural
networks with grammar parsing and program synthesis, learned by a novel
deduction--abduction strategy. In experiments, the proposed neural-symbolic
system demonstrates strong generalization capability and significantly
outperforms end-to-end neural methods like RNN and Transformer. The results
also indicate the significance of recursive priors for extrapolation on syntax
and semantics.
