A computational model of early language acquisition from audiovisual
  experiences of young infants
  Earlier research has suggested that human infants might use statistical
dependencies between speech and non-linguistic multimodal input to bootstrap
their language learning before they know how to segment words from running
speech. However, feasibility of this hypothesis in terms of real-world infant
experiences has remained unclear. This paper presents a step towards a more
realistic test of the multimodal bootstrapping hypothesis by describing a
neural network model that can learn word segments and their meanings from
referentially ambiguous acoustic input. The model is tested on recordings of
real infant-caregiver interactions using utterance-level labels for concrete
visual objects that were attended by the infant when caregiver spoke an
utterance containing the name of the object, and using random visual labels for
utterances during absence of attention. The results show that beginnings of
lexical knowledge may indeed emerge from individually ambiguous learning
scenarios. In addition, the hidden layers of the network show gradually
increasing selectivity to phonetic categories as a function of layer depth,
resembling models trained for phone recognition in a supervised manner.
