Enhanced Transfer Learning Through Medical Imaging and Patient
  Demographic Data Fusion
  In this work we examine the performance enhancement in classification of
medical imaging data when image features are combined with associated non-image
data. We compare the performance of eight state-of-the-art deep neural networks
in classification tasks when using only image features, compared to when these
are combined with patient metadata. We utilise transfer learning with networks
pretrained on ImageNet used directly as feature extractors and fine tuned on
the target domain. Our experiments show that performance can be significantly
enhanced with the inclusion of metadata and use interpretability methods to
identify which features lead to these enhancements. Furthermore, our results
indicate that the performance enhancement for natural medical imaging (e.g.
optical images) benefit most from direct use of pre-trained models, whereas non
natural images (e.g. representations of non imaging data) benefit most from
fine tuning pre-trained networks. These enhancements come at a negligible
additional cost in computation time, and therefore is a practical method for
other applications.
