SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction
  using Electronic Health Records
  Contrastive learning has demonstrated promising performance in image and text
domains either in a self-supervised or a supervised manner. In this work, we
extend the supervised contrastive learning framework to clinical risk
prediction problems based on longitudinal electronic health records (EHR). We
propose a general supervised contrastive loss $\mathcal{L}_{\text{Contrastive
Cross Entropy} } + \lambda \mathcal{L}_{\text{Supervised Contrastive
Regularizer}}$ for learning both binary classification (e.g. in-hospital
mortality prediction) and multi-label classification (e.g. phenotyping) in a
unified framework. Our supervised contrastive loss practices the key idea of
contrastive learning, namely, pulling similar samples closer and pushing
dissimilar ones apart from each other, simultaneously by its two components:
$\mathcal{L}_{\text{Contrastive Cross Entropy} }$ tries to contrast samples
with learned anchors which represent positive and negative clusters, and
$\mathcal{L}_{\text{Supervised Contrastive Regularizer}}$ tries to contrast
samples with each other according to their supervised labels. We propose two
versions of the above supervised contrastive loss and our experiments on
real-world EHR data demonstrate that our proposed loss functions show benefits
in improving the performance of strong baselines and even state-of-the-art
models on benchmarking tasks for clinical risk predictions. Our loss functions
work well with extremely imbalanced data which are common for clinical risk
prediction problems. Our loss functions can be easily used to replace (binary
or multi-label) cross-entropy loss adopted in existing clinical predictive
models. The Pytorch code is released at
\url{https://github.com/calvin-zcx/SCEHR}.
