Holographic Visualisation of Radiology Data and Automated Machine
  Learning-based Medical Image Segmentation
  Within this thesis we propose a platform for combining Augmented Reality (AR)
hardware with machine learning in a user-oriented pipeline, offering to the
medical staff an intuitive 3D visualization of volumetric Computed Tomography
(CT) and Magnetic Resonance Imaging (MRI) medical image segmentations inside
the AR headset, that does not need human intervention for loading, processing
and segmentation of medical images. The AR visualization, based on Microsoft
HoloLens, employs a modular and thus scalable frontend-backend architecture for
real-time visualizations on multiple AR headsets. As Convolutional Neural
Networks (CNNs) have lastly demonstrated superior performance for the machine
learning task of image semantic segmentation, the pipeline also includes a
fully automated CNN algorithm for the segmentation of the liver from CT scans.
The model is based on the Deep Retinal Image Understanding (DRIU) model which
is a Fully Convolutional Network with side outputs from feature maps with
different resolution, extracted at different stages of the network. The
algorithm is 2.5D which means that the input is a set of consecutive scan
slices. The experiments have been performed on the Liver Tumor Segmentation
Challenge (LiTS) dataset for liver segmentation and demonstrated good results
and flexibility. While multiple approaches exist in the domain, only few of
them have focused on overcoming the practical aspects which still largely hold
this technology away from the operating rooms. In line with this, we also are
next planning an evaluation from medical doctors and radiologists in a
real-world environment.
