Vector Learning for Cross Domain Representations
  Recently, generative adversarial networks have gained a lot of popularity for
image generation tasks. However, such models are associated with complex
learning mechanisms and demand very large relevant datasets. This work borrows
concepts from image and video captioning models to form an image generative
framework. The model is trained in a similar fashion as recurrent captioning
model and uses the learned weights for image generation. This is done in an
inverse direction, where the input is a caption and the output is an image. The
vector representation of the sentence and frames are extracted from an
encoder-decoder model which is initially trained on similar sentence and image
pairs. Our model conditions image generation on a natural language caption. We
leverage a sequence-to-sequence model to generate synthetic captions that have
the same meaning for having a robust image generation. One key advantage of our
method is that the traditional image captioning datasets can be used for
synthetic sentence paraphrases. Results indicate that images generated through
multiple captions are better at capturing the semantic meaning of the family of
captions.
