Distilling Knowledge from Deep Networks with Applications to Healthcare
  Domain
  Exponential growth in Electronic Healthcare Records (EHR) has resulted in new
opportunities and urgent needs for discovery of meaningful data-driven
representations and patterns of diseases in Computational Phenotyping research.
Deep Learning models have shown superior performance for robust prediction in
computational phenotyping tasks, but suffer from the issue of model
interpretability which is crucial for clinicians involved in decision-making.
In this paper, we introduce a novel knowledge-distillation approach called
Interpretable Mimic Learning, to learn interpretable phenotype features for
making robust prediction while mimicking the performance of deep learning
models. Our framework uses Gradient Boosting Trees to learn interpretable
features from deep learning models such as Stacked Denoising Autoencoder and
Long Short-Term Memory. Exhaustive experiments on a real-world clinical
time-series dataset show that our method obtains similar or better performance
than the deep learning models, and it provides interpretable phenotypes for
clinical decision making.
