Multi-modal, multi-task, multi-attention (M3) deep learning detection of
  reticular pseudodrusen: towards automated and accessible classification of
  age-related macular degeneration
  Objective Reticular pseudodrusen (RPD), a key feature of age-related macular
degeneration (AMD), are poorly detected by human experts on standard color
fundus photography (CFP) and typically require advanced imaging modalities such
as fundus autofluorescence (FAF). The objective was to develop and evaluate the
performance of a novel 'M3' deep learning framework on RPD detection. Materials
and Methods A deep learning framework M3 was developed to detect RPD presence
accurately using CFP alone, FAF alone, or both, employing >8000 CFP-FAF image
pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3
framework includes multi-modal (detection from single or multiple image
modalities), multi-task (training different tasks simultaneously to improve
generalizability), and multi-attention (improving ensembled feature
representation) operation. Performance on RPD detection was compared with
state-of-the-art deep learning models and 13 ophthalmologists; performance on
detection of two other AMD features (geographic atrophy and pigmentary
abnormalities) was also evaluated. Results For RPD detection, M3 achieved area
under receiver operating characteristic (AUROC) 0.832, 0.931, and 0.933 for CFP
alone, FAF alone, and both, respectively. M3 performance on CFP was very
substantially superior to human retinal specialists (median F1-score 0.644
versus 0.350). External validation (on Rotterdam Study, Netherlands)
demonstrated high accuracy on CFP alone (AUROC 0.965). The M3 framework also
accurately detected geographic atrophy and pigmentary abnormalities (AUROC
0.909 and 0.912, respectively), demonstrating its generalizability. Conclusion
This study demonstrates the successful development, robust evaluation, and
external validation of a novel deep learning framework that enables accessible,
accurate, and automated AMD diagnosis and prognosis.
