An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using
  Multimodal Data
  Accurate prognosis of a tumor can help doctors provide a proper course of
treatment and, therefore, save the lives of many. Traditional machine learning
algorithms have been eminently useful in crafting prognostic models in the last
few decades. Recently, deep learning algorithms have shown significant
improvement when developing diagnosis and prognosis solutions to different
healthcare problems. However, most of these solutions rely solely on either
imaging or clinical data. Utilizing patient tabular data such as demographics
and patient medical history alongside imaging data in a multimodal approach to
solve a prognosis task has started to gain more interest recently and has the
potential to create more accurate solutions. The main issue when using clinical
and imaging data to train a deep learning model is to decide on how to combine
the information from these sources. We propose a multimodal network that
ensembles deep multi-task logistic regression (MTLR), Cox proportional hazard
(CoxPH) and CNN models to predict prognostic outcomes for patients with head
and neck tumors using patients' clinical and imaging (CT and PET) data.
Features from CT and PET scans are fused and then combined with patients'
electronic health records for the prediction. The proposed model is trained and
tested on 224 and 101 patient records respectively. Experimental results show
that our proposed ensemble solution achieves a C-index of 0.72 on The HECKTOR
test set that saved us the first place in prognosis task of the HECKTOR
challenge. The full implementation based on PyTorch is available on
\url{https://github.com/numanai/BioMedIA-Hecktor2021}.
