Scalable Evaluation and Improvement of Document Set Expansion via Neural
  Positive-Unlabeled Learning
  We consider the situation in which a user has collected a small set of
documents on a cohesive topic, and they want to retrieve additional documents
on this topic from a large collection. Information Retrieval (IR) solutions
treat the document set as a query, and look for similar documents in the
collection. We propose to extend the IR approach by treating the problem as an
instance of positive-unlabeled (PU) learning -- i.e., learning binary
classifiers from only positive and unlabeled data, where the positive data
corresponds to the query documents, and the unlabeled data is the results
returned by the IR engine. Utilizing PU learning for text with big neural
networks is a largely unexplored field. We discuss various challenges in
applying PU learning to the setting, including an unknown class prior,
extremely imbalanced data and large-scale accurate evaluation of models, and we
propose solutions and empirically validate them. We demonstrate the
effectiveness of the method using a series of experiments of retrieving PubMed
abstracts adhering to fine-grained topics. We demonstrate improvements over the
base IR solution and other baselines.
