A Question-Centric Model for Visual Question Answering in Medical
  Imaging
  Deep learning methods have proven extremely effective at performing a variety
of medical image analysis tasks. With their potential use in clinical routine,
their lack of transparency has however been one of their few weak points,
raising concerns regarding their behavior and failure modes. While most
research to infer model behavior has focused on indirect strategies that
estimate prediction uncertainties and visualize model support in the input
image space, the ability to explicitly query a prediction model regarding its
image content offers a more direct way to determine the behavior of trained
models. To this end, we present a novel Visual Question Answering approach that
allows an image to be queried by means of a written question. Experiments on a
variety of medical and natural image datasets show that by fusing image and
question features in a novel way, the proposed approach achieves an equal or
higher accuracy compared to current methods.
