Multimodal Learning for Cardiovascular Risk Prediction using EHR Data
  Electronic health records (EHRs) contain structured and unstructured data of
significant clinical and research value. Various machine learning approaches
have been developed to employ information in EHRs for risk prediction. The
majority of these attempts, however, focus on structured EHR fields and lose
the vast amount of information in the unstructured texts. To exploit the
potential information captured in EHRs, in this study we propose a multimodal
recurrent neural network model for cardiovascular risk prediction that
integrates both medical texts and structured clinical information. The proposed
multimodal bidirectional long short-term memory (BiLSTM) model concatenates
word embeddings to classical clinical predictors before applying them to a
final fully connected neural network. In the experiments, we compare
performance of different deep neural network (DNN) architectures including
convolutional neural network and long short-term memory in scenarios of using
clinical variables and chest X-ray radiology reports. Evaluated on a data set
of real world patients with manifest vascular disease or at high-risk for
cardiovascular disease, the proposed BiLSTM model demonstrates state-of-the-art
performance and outperforms other DNN baseline architectures.
