Self-supervised Fine-tuning for Correcting Super-Resolution
  Convolutional Neural Networks
  While Convolutional Neural Networks (CNNs) trained for image and video
super-resolution (SR) regularly achieve new state-of-the-art performance, they
also suffer from significant drawbacks. One of their limitations is their lack
of robustness to unseen image formation models during training. Other
limitations include the generation of artifacts and hallucinated content when
training Generative Adversarial Networks (GANs) for SR. While the Deep Learning
literature focuses on presenting new training schemes and settings to resolve
these various issues, we show that one can avoid training and correct for SR
results with a fully self-supervised fine-tuning approach. More specifically,
at test time, given an image and its known image formation model, we fine-tune
the parameters of the trained network and iteratively update them using a data
fidelity loss. We apply our fine-tuning algorithm on multiple image and video
SR CNNs and show that it can successfully correct for a sub-optimal SR solution
by entirely relying on internal learning at test time. We apply our method on
the problem of fine-tuning for unseen image formation models and on removal of
artifacts introduced by GANs.
