Hear No Evil: Towards Adversarial Robustness of Automatic Speech
  Recognition via Multi-Task Learning
  As automatic speech recognition (ASR) systems are now being widely deployed
in the wild, the increasing threat of adversarial attacks raises serious
questions about the security and reliability of using such systems. On the
other hand, multi-task learning (MTL) has shown success in training models that
can resist adversarial attacks in the computer vision domain. In this work, we
investigate the impact of performing such multi-task learning on the
adversarial robustness of ASR models in the speech domain. We conduct extensive
MTL experimentation by combining semantically diverse tasks such as accent
classification and ASR, and evaluate a wide range of adversarial settings. Our
thorough analysis reveals that performing MTL with semantically diverse tasks
consistently makes it harder for an adversarial attack to succeed. We also
discuss in detail the serious pitfalls and their related remedies that have a
significant impact on the robustness of MTL models. Our proposed MTL approach
shows considerable absolute improvements in adversarially targeted WER ranging
from 17.25 up to 59.90 compared to single-task learning baselines (attention
decoder and CTC respectively). Ours is the first in-depth study that uncovers
adversarial robustness gains from multi-task learning for ASR.
