On the Role of Visual Cues in Audiovisual Speech Enhancement
  We present an introspection of an audiovisual speech enhancement model. In
particular, we focus on interpreting how a neural audiovisual speech
enhancement model uses visual cues to improve the quality of the target speech
signal. We show that visual cues provide not only high-level information about
speech activity, i.e., speech/silence, but also fine-grained visual information
about the place of articulation. One byproduct of this finding is that the
learned visual embeddings can be used as features for other visual speech
applications. We demonstrate the effectiveness of the learned visual embeddings
for classifying visemes (the visual analogy to phonemes). Our results provide
insight into important aspects of audiovisual speech enhancement and
demonstrate how such models can be used for self-supervision tasks for visual
speech applications.
