Low Bandwidth Video-Chat Compression using Deep Generative Models
  To unlock video chat for hundreds of millions of people hindered by poor
connectivity or unaffordable data costs, we propose to authentically
reconstruct faces on the receiver's device using facial landmarks extracted at
the sender's side and transmitted over the network. In this context, we discuss
and evaluate the benefits and disadvantages of several deep adversarial
approaches. In particular, we explore quality and bandwidth trade-offs for
approaches based on static landmarks, dynamic landmarks or segmentation maps.
We design a mobile-compatible architecture based on the first order animation
model of Siarohin et al. In addition, we leverage SPADE blocks to refine
results in important areas such as the eyes and lips. We compress the networks
down to about 3MB, allowing models to run in real time on iPhone 8 (CPU). This
approach enables video calling at a few kbits per second, an order of magnitude
lower than currently available alternatives.
