FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection
  The strong relation between face and voice can aid active speaker detection
systems when faces are visible, even in difficult settings, when the face of a
speaker is not clear or when there are several people in the same scene. By
being capable of estimating the frontal facial representation of a person from
his/her speech, it becomes easier to determine whether he/she is a potential
candidate for being classified as an active speaker, even in challenging cases
in which no mouth movement is detected from any person in that same scene. By
incorporating a face-voice association neural network into an existing
state-of-the-art active speaker detection model, we introduce FaVoA (Face-Voice
Association Ambiguous Speaker Detector), a neural network model that can
correctly classify particularly ambiguous scenarios. FaVoA not only finds
positive associations, but helps to rule out non-matching face-voice
associations, where a face does not match a voice. Its use of a
gated-bimodal-unit architecture for the fusion of those models offers a way to
quantitatively determine how much each modality contributes to the
classification.
