Acoustic Neighbor Embeddings
  This paper proposes a novel acoustic word embedding called Acoustic Neighbor
Embeddings where speech or text of arbitrary length are mapped to a vector
space of fixed, reduced dimensions by adapting stochastic neighbor embedding
(SNE) to sequential inputs. The Euclidean distance between coordinates in the
embedding space reflects the phonetic confusability between their corresponding
sequences. Two encoder neural networks are trained: an acoustic encoder that
accepts speech signals in the form of frame-wise subword posterior
probabilities obtained from an acoustic model and a text encoder that accepts
text in the form of subword transcriptions. Compared to a triplet loss
criterion, the proposed method is shown to have more effective gradients for
neural network training. Experimentally, it also gives more accurate results
with low-dimensional embeddings when the two encoder networks are used in
tandem in a word (name) recognition task, and when the text encoder network is
used standalone in an approximate phonetic matching task. In particular, in an
isolated name recognition task depending solely on Euclidean nearest-neighbor
search between the proposed embedding vectors, the recognition accuracy is
identical to that of conventional finite state transducer(FST)-based decoding
using test data with up to 1 million names in the vocabulary and 40 dimensions
in the embeddings.
