SD-RSIC: Summarization Driven Deep Remote Sensing Image Captioning
  Deep neural networks (DNNs) have been recently found popular for image
captioning problems in remote sensing (RS). Existing DNN based approaches rely
on the availability of a training set made up of a high number of RS images
with their captions. However, captions of training images may contain redundant
information (they can be repetitive or semantically similar to each other),
resulting in information deficiency while learning a mapping from the image
domain to the language domain. To overcome this limitation, in this paper, we
present a novel Summarization Driven Remote Sensing Image Captioning (SD-RSIC)
approach. The proposed approach consists of three main steps. The first step
obtains the standard image captions by jointly exploiting convolutional neural
networks (CNNs) with long short-term memory (LSTM) networks. The second step,
unlike the existing RS image captioning methods, summarizes the ground-truth
captions of each training image into a single caption by exploiting sequence to
sequence neural networks and eliminates the redundancy present in the training
set. The third step automatically defines the adaptive weights associated to
each RS image to combine the standard captions with the summarized captions
based on the semantic content of the image. This is achieved by a novel
adaptive weighting strategy defined in the context of LSTM networks.
Experimental results obtained on the RSCID, UCM-Captions and Sydney-Captions
datasets show the effectiveness of the proposed approach compared to the
state-of-the-art RS image captioning approaches. The code of the proposed
approach is publicly available at
https://gitlab.tubit.tu-berlin.de/rsim/SD-RSIC.
