Neural Language Codes for Multilingual Acoustic Models
  Multilingual Speech Recognition is one of the most costly AI problems,
because each language (7,000+) and even different accents require their own
acoustic models to obtain best recognition performance. Even though they all
use the same phoneme symbols, each language and accent imposes its own coloring
or "twang". Many adaptive approaches have been proposed, but they require
further training, additional data and generally are inferior to monolingually
trained models. In this paper, we propose a different approach that uses a
large multilingual model that is \emph{modulated} by the codes generated by an
ancillary network that learns to code useful differences between the "twangs"
or human language.
  We use Meta-Pi networks to have one network (the language code net) gate the
activity of neurons in another (the acoustic model nets). Our results show that
during recognition multilingual Meta-Pi networks quickly adapt to the proper
language coloring without retraining or new data, and perform better than
monolingually trained networks. The model was evaluated by training acoustic
modeling nets and modulating language code nets jointly and optimize them for
best recognition performance.
