Perceptually-inspired super-resolution of compressed videos
  Spatial resolution adaptation is a technique which has often been employed in
video compression to enhance coding efficiency. This approach encodes a lower
resolution version of the input video and reconstructs the original resolution
during decoding. Instead of using conventional up-sampling filters, recent work
has employed advanced super-resolution methods based on convolutional neural
networks (CNNs) to further improve reconstruction quality. These approaches are
usually trained to minimise pixel-based losses such as Mean-Squared Error
(MSE), despite the fact that this type of loss metric does not correlate well
with subjective opinions. In this paper, a perceptually-inspired
super-resolution approach (M-SRGAN) is proposed for spatial up-sampling of
compressed video using a modified CNN model, which has been trained using a
generative adversarial network (GAN) on compressed content with perceptual loss
functions. The proposed method was integrated with HEVC HM 16.20, and has been
evaluated on the JVET Common Test Conditions (UHD test sequences) using the
Random Access configuration. The results show evident perceptual quality
improvement over the original HM 16.20, with an average bitrate saving of 35.6%
(Bj{\o}ntegaard Delta measurement) based on a perceptual quality metric, VMAF.
