DeepSUM: Deep neural network for Super-resolution of Unregistered
  Multitemporal images
  Recently, convolutional neural networks (CNN) have been successfully applied
to many remote sensing problems. However, deep learning techniques for
multi-image super-resolution from multitemporal unregistered imagery have
received little attention so far. This work proposes a novel CNN-based
technique that exploits both spatial and temporal correlations to combine
multiple images. This novel framework integrates the spatial registration task
directly inside the CNN, and allows to exploit the representation learning
capabilities of the network to enhance registration accuracy. The entire
super-resolution process relies on a single CNN with three main stages: shared
2D convolutions to extract high-dimensional features from the input images; a
subnetwork proposing registration filters derived from the high-dimensional
feature representations; 3D convolutions for slow fusion of the features from
multiple images. The whole network can be trained end-to-end to recover a
single high resolution image from multiple unregistered low resolution images.
The method presented in this paper is the winner of the PROBA-V
super-resolution challenge issued by the European Space Agency.
