GreaseLM: Graph REASoning Enhanced Language Models for Question
  Answering
  Answering complex questions about textual narratives requires reasoning over
both stated context and the world knowledge that underlies it. However,
pretrained language models (LM), the foundation of most modern QA systems, do
not robustly represent latent relationships between concepts, which is
necessary for reasoning. While knowledge graphs (KG) are often used to augment
LMs with structured representations of world knowledge, it remains an open
question how to effectively fuse and reason over the KG representations and the
language context, which provides situational constraints and nuances. In this
work, we propose GreaseLM, a new model that fuses encoded representations from
pretrained LMs and graph neural networks over multiple layers of modality
interaction operations. Information from both modalities propagates to the
other, allowing language context representations to be grounded by structured
world knowledge, and allowing linguistic nuances (e.g., negation, hedging) in
the context to inform the graph representations of knowledge. Our results on
three benchmarks in the commonsense reasoning (i.e., CommonsenseQA, OpenbookQA)
and medical question answering (i.e., MedQA-USMLE) domains demonstrate that
GreaseLM can more reliably answer questions that require reasoning over both
situational constraints and structured knowledge, even outperforming models 8x
larger.
