Spatially-Adaptive Image Restoration using Distortion-Guided Networks
  We present a general learning-based solution for restoring images suffering
from spatially-varying degradations. Prior approaches are typically
degradation-specific and employ the same processing across different images and
different pixels within. However, we hypothesize that such spatially rigid
processing is suboptimal for simultaneously restoring the degraded pixels as
well as reconstructing the clean regions of the image. To overcome this
limitation, we propose SPAIR, a network design that harnesses
distortion-localization information and dynamically adjusts computation to
difficult regions in the image. SPAIR comprises of two components, (1) a
localization network that identifies degraded pixels, and (2) a restoration
network that exploits knowledge from the localization network in filter and
feature domain to selectively and adaptively restore degraded pixels. Our key
idea is to exploit the non-uniformity of heavy degradations in spatial-domain
and suitably embed this knowledge within distortion-guided modules performing
sparse normalization, feature extraction and attention. Our architecture is
agnostic to physical formation model and generalizes across several types of
spatially-varying degradations. We demonstrate the efficacy of SPAIR
individually on four restoration tasks-removal of rain-streaks, raindrops,
shadows and motion blur. Extensive qualitative and quantitative comparisons
with prior art on 11 benchmark datasets demonstrate that our
degradation-agnostic network design offers significant performance gains over
state-of-the-art degradation-specific architectures. Code available at
https://github.com/human-analysis/spatially-adaptive-image-restoration.
