Visual Question Answering with Prior Class Semantics
  We present a novel mechanism to embed prior knowledge in a model for visual
question answering. The open-set nature of the task is at odds with the
ubiquitous approach of training of a fixed classifier. We show how to exploit
additional information pertaining to the semantics of candidate answers. We
extend the answer prediction process with a regression objective in a semantic
space, in which we project candidate answers using prior knowledge derived from
word embeddings. We perform an extensive study of learned representations with
the GQA dataset, revealing that important semantic information is captured in
the relations between embeddings in the answer space. Our method brings
improvements in consistency and accuracy over a range of question types.
Experiments with novel answers, unseen during training, indicate the method's
potential for open-set prediction.
