Viability of machine learning to reduce workload in systematic review
  screenings in the health sciences: a working paper
  Systematic reviews, which summarize and synthesize all the current research
in a specific topic, are a crucial component to academia. They are especially
important in the biomedical and health sciences, where they synthesize the
state of medical evidence and conclude the best course of action for various
diseases, pathologies, and treatments. Due to the immense amount of literature
that exists, as well as the output rate of research, reviewing abstracts can be
a laborious process. Automation may be able to significantly reduce this
workload. Of course, such classifications are not easily automated due to the
peculiar nature of written language. Machine learning may be able to help. This
paper explored the viability and effectiveness of using machine learning
modelling to classify abstracts according to specific exclusion/inclusion
criteria, as would be done in the first stage of a systematic review. The
specific task was performing the classification of deciding whether an abstract
is a randomized control trial (RCT) or not, a very common classification made
in systematic reviews in the healthcare field. Random training/testing splits
of an n=2042 dataset of labelled abstracts were repeatedly created (1000 times
in total), with a model trained and tested on each of these instances. A Bayes
classifier as well as an SVM classifier were used, and compared to non-machine
learning, simplistic approaches to textual classification. An SVM classifier
was seen to be highly effective, yielding a 90% accuracy, as well as an F1
score of 0.84, and yielded a potential workload reduction of 70%. This shows
that machine learning has the potential to significantly revolutionize the
abstract screening process in healthcare systematic reviews.
