Towards Practical Lipreading with Distilled and Efficient Models
  Lipreading has witnessed a lot of progress due to the resurgence of neural
networks. Recent works have placed emphasis on aspects such as improving
performance by finding the optimal architecture or improving generalization.
However, there is still a significant gap between the current methodologies and
the requirements for an effective deployment of lipreading in practical
scenarios. In this work, we propose a series of innovations that significantly
bridge that gap: first, we raise the state-of-the-art performance by a wide
margin on LRW and LRW-1000 to 88.5% and 46.6%, respectively using
self-distillation. Secondly, we propose a series of architectural changes,
including a novel Depthwise Separable Temporal Convolutional Network (DS-TCN)
head, that slashes the computational cost to a fraction of the (already quite
efficient) original model. Thirdly, we show that knowledge distillation is a
very effective tool for recovering performance of the lightweight models. This
results in a range of models with different accuracy-efficiency trade-offs.
However, our most promising lightweight models are on par with the current
state-of-the-art while showing a reduction of 8.2x and 3.9x in terms of
computational cost and number of parameters, respectively, which we hope will
enable the deployment of lipreading models in practical applications.
