Speaker-Independent Speech-Driven Visual Speech Synthesis using
  Domain-Adapted Acoustic Models
  Speech-driven visual speech synthesis involves mapping features extracted
from acoustic speech to the corresponding lip animation controls for a face
model. This mapping can take many forms, but a powerful approach is to use deep
neural networks (DNNs). However, a limitation is the lack of synchronized
audio, video, and depth data required to reliably train the DNNs, especially
for speaker-independent models. In this paper, we investigate adapting an
automatic speech recognition (ASR) acoustic model (AM) for the visual speech
synthesis problem. We train the AM on ten thousand hours of audio-only data.
The AM is then adapted to the visual speech synthesis domain using ninety hours
of synchronized audio-visual speech. Using a subjective assessment test, we
compared the performance of the AM-initialized DNN to one with a random
initialization. The results show that viewers significantly prefer animations
generated from the AM-initialized DNN than the ones generated using the
randomly initialized model. We conclude that visual speech synthesis can
significantly benefit from the powerful representation of speech in the ASR
acoustic models.
