Nodule2vec: a 3D Deep Learning System for Pulmonary Nodule Retrieval
  Using Semantic Representation
  Content-based retrieval supports a radiologist decision making process by
presenting the doctor the most similar cases from the database containing both
historical diagnosis and further disease development history. We present a deep
learning system that transforms a 3D image of a pulmonary nodule from a CT scan
into a low-dimensional embedding vector. We demonstrate that such a vector
representation preserves semantic information about the nodule and offers a
viable approach for content-based image retrieval (CBIR). We discuss the
theoretical limitations of the available datasets and overcome them by applying
transfer learning of the state-of-the-art lung nodule detection model. We
evaluate the system using the LIDC-IDRI dataset of thoracic CT scans. We devise
a similarity score and show that it can be utilized to measure similarity 1)
between annotations of the same nodule by different radiologists and 2) between
the query nodule and the top four CBIR results. A comparison between doctors
and algorithm scores suggests that the benefit provided by the system to the
radiologist end-user is comparable to obtaining a second radiologist's opinion.
