Mosaic Super-resolution via Sequential Feature Pyramid Networks
  Advances in the design of multi-spectral cameras have led to great interests
in a wide range of applications, from astronomy to autonomous driving. However,
such cameras inherently suffer from a trade-off between the spatial and
spectral resolution. In this paper, we propose to address this limitation by
introducing a novel method to carry out super-resolution on raw mosaic images,
multi-spectral or RGB Bayer, captured by modern real-time single-shot mosaic
sensors. To this end, we design a deep super-resolution architecture that
benefits from a sequential feature pyramid along the depth of the network.
This, in fact, is achieved by utilizing a convolutional LSTM (ConvLSTM) to
learn the inter-dependencies between features at different receptive fields.
Additionally, by investigating the effect of different attention mechanisms in
our framework, we show that a ConvLSTM inspired module is able to provide
superior attention in our context. Our extensive experiments and analyses
evidence that our approach yields significant super-resolution quality,
outperforming current state-of-the-art mosaic super-resolution methods on both
Bayer and multi-spectral images. Additionally, to the best of our knowledge,
our method is the first specialized method to super-resolve mosaic images,
whether it be multi-spectral or Bayer.
