Distilling Information from a Flood: A Possibility for the Use of
  Meta-Analysis and Systematic Review in Machine Learning Research
  The current flood of information in all areas of machine learning research,
from computer vision to reinforcement learning, has made it difficult to make
aggregate scientific inferences. It can be challenging to distill a myriad of
similar papers into a set of useful principles, to determine which new
methodologies to use for a particular application, and to be confident that one
has compared against all relevant related work when developing new ideas.
However, such a rapidly growing body of research literature is a problem that
other fields have already faced - in particular, medicine and epidemiology. In
those fields, systematic reviews and meta-analyses have been used exactly for
dealing with these issues and it is not uncommon for entire journals to be
dedicated to such analyses. Here, we suggest the field of machine learning
might similarly benefit from meta-analysis and systematic review, and we
encourage further discussion and development along this direction.
