Performance of a deep learning system for detection of referable diabetic retinopathy in real clinical settings
Background: To determine the ability of a commercially available deep
learning system, RetCAD v.1.3.1 (Thirona, Nijmegen, The Netherlands) for the
automatic detection of referable diabetic retinopathy (DR) on a dataset of
colour fundus images acquired during routine clinical practice in a tertiary
hospital screening program, analyzing the reduction of workload that can be
released incorporating this artificial intelligence-based technology. Methods:
Evaluation of the software was performed on a dataset of 7195 nonmydriatic
fundus images from 6325 eyes of 3189 diabetic patients attending our screening
program between February to December of 2019. The software generated a DR
severity score for each colour fundus image which was combined into an
eye-level score. This score was then compared with a reference standard as set
by a human expert using receiver operating characteristic (ROC) curve analysis.
Results: The artificial intelligence (AI) software achieved an area under the
ROC curve (AUC) value of 0.988 [0.981:0.993] for the detection of referable DR.
At the proposed operating point, the sensitivity of the RetCAD software for DR
is 90.53% and specificity is 97.13%. A workload reduction of 96% could be
achieved at the cost of only 6 false negatives. Conclusions: The AI software
correctly identified the vast majority of referable DR cases, with a workload
reduction of 96% of the cases that would need to be checked, while missing
almost no true cases, so it may therefore be used as an instrument for triage.