Effective Representations of Clinical Notes
  Clinical notes are a rich source of information about patient state. However,
using them to predict clinical events with machine learning models is
challenging. They are very high dimensional, sparse and have complex structure.
Furthermore, training data is often scarce because it is expensive to obtain
reliable labels for many clinical events. These difficulties have traditionally
been addressed by manual feature engineering encoding task specific domain
knowledge. We explored the use of neural networks and transfer learning to
learn representations of clinical notes that are useful for predicting future
clinical events of interest, such as all causes mortality, inpatient
admissions, and emergency room visits. Our data comprised 2.7 million notes and
115 thousand patients at Stanford Hospital. We used the learned
representations, along with commonly used bag of words and topic model
representations, as features for predictive models of clinical events. We
evaluated the effectiveness of these representations with respect to the
performance of the models trained on small datasets. Models using the neural
network derived representations performed significantly better than models
using the baseline representations with small ($N < 1000$) training datasets.
The learned representations offer significant performance gains over commonly
used baseline representations for a range of predictive modeling tasks and
cohort sizes, offering an effective alternative to task specific feature
engineering when plentiful labeled training data is not available.
