ChartNet: Visual Reasoning over Statistical Charts using MAC-Networks
  Despite the improvements in perception accuracies brought about via deep
learning, developing systems combining accurate visual perception with the
ability to reason over the visual percepts remains extremely challenging. A
particular application area of interest from an accessibility perspective is
that of reasoning over statistical charts such as bar and pie charts. To this
end, we formulate the problem of reasoning over statistical charts as a
classification task using MAC-Networks to give answers from a predefined
vocabulary of generic answers. Additionally, we enhance the capabilities of
MAC-Networks to give chart-specific answers to open-ended questions by
replacing the classification layer by a regression layer to localize the
textual answers present over the images. We call our network ChartNet, and
demonstrate its efficacy on predicting both in vocabulary and out of vocabulary
answers. To test our methods, we generated our own dataset of statistical chart
images and corresponding question answer pairs. Results show that ChartNet
consistently outperform other state-of-the-art methods on reasoning over these
questions and may be a viable candidate for applications containing images of
statistical charts.
