Multi-task Ensembles with Crowdsourced Features Improve Skin Lesion
  Diagnosis
  Machine learning has a recognised need for large amounts of annotated data.
Due to the high cost of expert annotations, crowdsourcing, where non-experts
are asked to label or outline images, has been proposed as an alternative.
Although many promising results are reported, the quality of diagnostic
crowdsourced labels is still unclear. We propose to address this by instead
asking the crowd about visual features of the images, which can be provided
more intuitively, and by using these features in a multi-task learning
framework through ensemble strategies. We compare our proposed approach to a
baseline model with a set of 2000 skin lesions from the ISIC 2017 challenge
dataset. The baseline model only predicts a binary label from the skin lesion
image, while our multi-task model also predicts one of the following features:
asymmetry of the lesion, border irregularity and color. We show that multi-task
models with individual crowdsourced features have limited effect on the model,
but when combined in an ensembles, leads to improved generalisation. The area
under the receiver operating characteristic curve is 0.794 for the baseline
model and 0.811 and 0.808 for multi-task ensembles respectively. Finally, we
discuss the findings, identify some limitations and recommend directions for
further research. The code of the models is available at
https://github.com/raumannsr/hints_crowd.
