Supervised Reinforcement Learning with Recurrent Neural Network for
  Dynamic Treatment Recommendation
  Dynamic treatment recommendation systems based on large-scale electronic
health records (EHRs) become a key to successfully improve practical clinical
outcomes. Prior relevant studies recommend treatments either use supervised
learning (e.g. matching the indicator signal which denotes doctor
prescriptions), or reinforcement learning (e.g. maximizing evaluation signal
which indicates cumulative reward from survival rates). However, none of these
studies have considered to combine the benefits of supervised learning and
reinforcement learning. In this paper, we propose Supervised Reinforcement
Learning with Recurrent Neural Network (SRL-RNN), which fuses them into a
synergistic learning framework. Specifically, SRL-RNN applies an off-policy
actor-critic framework to handle complex relations among multiple medications,
diseases and individual characteristics. The "actor" in the framework is
adjusted by both the indicator signal and evaluation signal to ensure effective
prescription and low mortality. RNN is further utilized to solve the
Partially-Observed Markov Decision Process (POMDP) problem due to the lack of
fully observed states in real world applications. Experiments on the publicly
real-world dataset, i.e., MIMIC-3, illustrate that our model can reduce the
estimated mortality, while providing promising accuracy in matching doctors'
prescriptions.
