Should we hard-code the recurrence concept or learn it instead ?
  Exploring the Transformer architecture for Audio-Visual Speech Recognition
  The audio-visual speech fusion strategy AV Align has shown significant
performance improvements in audio-visual speech recognition (AVSR) on the
challenging LRS2 dataset. Performance improvements range between 7% and 30%
depending on the noise level when leveraging the visual modality of speech in
addition to the auditory one. This work presents a variant of AV Align where
the recurrent Long Short-term Memory (LSTM) computation block is replaced by
the more recently proposed Transformer block. We compare the two methods,
discussing in greater detail their strengths and weaknesses. We find that
Transformers also learn cross-modal monotonic alignments, but suffer from the
same visual convergence problems as the LSTM model, calling for a deeper
investigation into the dominant modality problem in machine learning.
