Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning"
  Visual reasoning tasks such as visual question answering (VQA) require an
interplay of visual perception with reasoning about the question semantics
grounded in perception. However, recent advances in this area are still
primarily driven by perception improvements (e.g. scene graph generation)
rather than reasoning. Neuro-symbolic models such as Neural Module Networks
bring the benefits of compositional reasoning to VQA, but they are still
entangled with visual representation learning, and thus neural reasoning is
hard to improve and assess on its own. To address this, we propose (1) a
framework to isolate and evaluate the reasoning aspect of VQA separately from
its perception, and (2) a novel top-down calibration technique that allows the
model to answer reasoning questions even with imperfect perception. To this
end, we introduce a differentiable first-order logic formalism for VQA that
explicitly decouples question answering from visual perception. On the
challenging GQA dataset, this framework is used to perform in-depth,
disentangled comparisons between well-known VQA models leading to informative
insights regarding the participating models as well as the task.
