The pitfalls of using open data to develop deep learning solutions for
  COVID-19 detection in chest X-rays
  Since the emergence of COVID-19, deep learning models have been developed to
identify COVID-19 from chest X-rays. With little to no direct access to
hospital data, the AI community relies heavily on public data comprising
numerous data sources. Model performance results have been exceptional when
training and testing on open-source data, surpassing the reported capabilities
of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study
impactful models are trained on a widely used open-source data and tested on an
external test set and a hospital dataset, for the task of classifying chest
X-rays into one of three classes: COVID-19, non-COVID pneumonia and
no-pneumonia. Classification performance of the models investigated is
evaluated through ROC curves, confusion matrices and standard classification
metrics. Explainability modules are implemented to explore the image features
most important to classification. Data analysis and model evaluations show that
the popular open-source dataset COVIDx is not representative of the real
clinical problem and that results from testing on this are inflated. Dependence
on open-source data can leave models vulnerable to bias and confounding
variables, requiring careful analysis to develop clinically useful/viable AI
tools for COVID-19 detection in chest X-rays.
