BIGPrior: Towards Decoupling Learned Prior Hallucination and Data
  Fidelity in Image Restoration
  Classic image-restoration algorithms use a variety of priors, either
implicitly or explicitly. Their priors are hand-designed and their
corresponding weights are heuristically assigned. Hence, deep learning methods
often produce superior image restoration quality. Deep networks are, however,
capable of inducing strong and hardly predictable hallucinations. Networks
implicitly learn to be jointly faithful to the observed data while learning an
image prior; and the separation of original data and hallucinated data
downstream is then not possible. This limits their wide-spread adoption in
image restoration. Furthermore, it is often the hallucinated part that is
victim to degradation-model overfitting.
  We present an approach with decoupled network-prior based hallucination and
data fidelity terms. We refer to our framework as the Bayesian Integration of a
Generative Prior (BIGPrior). Our method is rooted in a Bayesian framework and
tightly connected to classic restoration methods. In fact, it can be viewed as
a generalization of a large family of classic restoration algorithms. We use
network inversion to extract image prior information from a generative network.
We show that, on image colorization, inpainting and denoising, our framework
consistently improves the inversion results. Our method, though partly reliant
on the quality of the generative network inversion, is competitive with
state-of-the-art supervised and task-specific restoration methods. It also
provides an additional metric that sets forth the degree of prior reliance per
pixel relative to data fidelity.
