Online Inference for Relation Extraction with a Reduced Feature Set
  Access to web-scale corpora is gradually bringing robust automatic knowledge
base creation and extension within reach. To exploit these large
unannotated---and extremely difficult to annotate---corpora, unsupervised
machine learning methods are required. Probabilistic models of text have
recently found some success as such a tool, but scalability remains an obstacle
in their application, with standard approaches relying on sampling schemes that
are known to be difficult to scale. In this report, we therefore present an
empirical assessment of the sublinear time sparse stochastic variational
inference (SSVI) scheme applied to RelLDA. We demonstrate that online inference
leads to relatively strong qualitative results but also identify some of its
pathologies---and those of the model---which will need to be overcome if SSVI
is to be used for large-scale relation extraction.
