Single Pair Cross-Modality Super Resolution
  Non-visual imaging sensors are widely used in the industry for different
purposes. Those sensors are more expensive than visual (RGB) sensors, and
usually produce images with lower resolution. To this end, Cross-Modality
Super-Resolution methods were introduced, where an RGB image of a
high-resolution assists in increasing the resolution of the low-resolution
modality. However, fusing images from different modalities is not a trivial
task; the output must be artifact-free and remain loyal to the characteristics
of the target modality. Moreover, the input images are never perfectly aligned,
which results in further artifacts during the fusion process.
  We present CMSR, a deep network for Cross-Modality Super-Resolution, which
unlike previous methods, is designed to deal with weakly aligned images. The
network is trained on the two input images only, learns their internal
statistics and correlations, and applies them to up-sample the target modality.
CMSR contains an internal transformer that is trained on-the-fly together with
the up-sampling process itself, without explicit supervision. We show that CMSR
succeeds to increase the resolution of the input image, gaining valuable
information from its RGB counterpart, yet in a conservative way, without
introducing artifacts or irrelevant details.
