From phonemes to images: levels of representation in a recurrent neural
  model of visually-grounded language learning
  We present a model of visually-grounded language learning based on stacked
gated recurrent neural networks which learns to predict visual features given
an image description in the form of a sequence of phonemes. The learning task
resembles that faced by human language learners who need to discover both
structure and meaning from noisy and ambiguous data across modalities. We show
that our model indeed learns to predict features of the visual context given
phonetically transcribed image descriptions, and show that it represents
linguistic information in a hierarchy of levels: lower layers in the stack are
comparatively more sensitive to form, whereas higher layers are more sensitive
to meaning.
