Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic
  Images via GANs
  Convolutional Neural Networks (CNN)-based approaches have shown promising
results in pansharpening of satellite images in recent years. However, they
still exhibit limitations in producing high-quality pansharpening outputs. To
that end, we propose a new self-supervised learning framework, where we treat
pansharpening as a colorization problem, which brings an entirely novel
perspective and solution to the problem compared to existing methods that base
their solution solely on producing a super-resolution version of the
multispectral image. Whereas CNN-based methods provide a reduced resolution
panchromatic image as input to their model along with reduced resolution
multispectral images, hence learn to increase their resolution together, we
instead provide the grayscale transformed multispectral image as input, and
train our model to learn the colorization of the grayscale input. We further
address the fixed downscale ratio assumption during training, which does not
generalize well to the full-resolution scenario. We introduce a noise injection
into the training by randomly varying the downsampling ratios. Those two
critical changes, along with the addition of adversarial training in the
proposed PanColorization Generative Adversarial Networks (PanColorGAN)
framework, help overcome the spatial detail loss and blur problems that are
observed in CNN-based pansharpening. The proposed approach outperforms the
previous CNN-based and traditional methods as demonstrated in our experiments.
