D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application
  to Skin Lesion Segmentation
  Medical image segmentation annotations suffer from inter- and intra-observer
variations even among experts due to intrinsic differences in human annotators
and ambiguous boundaries. Leveraging a collection of annotators' opinions for
an image is an interesting way of estimating a gold standard. Although training
deep models in a supervised setting with a single annotation per image has been
extensively studied, generalizing their training to work with datasets
containing multiple annotations per image remains a fairly unexplored problem.
In this paper, we propose an approach to handle annotators' disagreements when
training a deep model. To this end, we propose an ensemble of Bayesian fully
convolutional networks (FCNs) for the segmentation task by considering two
major factors in the aggregation of multiple ground truth annotations: (1)
handling contradictory annotations in the training data originating from
inter-annotator disagreements and (2) improving confidence calibration through
the fusion of base models' predictions. We demonstrate the superior performance
of our approach on the ISIC Archive and explore the generalization performance
of our proposed method by cross-dataset evaluation on the PH2 and DermoFit
datasets.
