Is Deep Reinforcement Learning Ready for Practical Applications in
  Healthcare? A Sensitivity Analysis of Duel-DDQN for Hemodynamic Management in
  Sepsis Patients
  The potential of Reinforcement Learning (RL) has been demonstrated through
successful applications to games such as Go and Atari. However, while it is
straightforward to evaluate the performance of an RL algorithm in a game
setting by simply using it to play the game, evaluation is a major challenge in
clinical settings where it could be unsafe to follow RL policies in practice.
Thus, understanding sensitivity of RL policies to the host of decisions made
during implementation is an important step toward building the type of trust in
RL required for eventual clinical uptake. In this work, we perform a
sensitivity analysis on a state-of-the-art RL algorithm (Dueling Double Deep
Q-Networks)applied to hemodynamic stabilization treatment strategies for septic
patients in the ICU. We consider sensitivity of learned policies to input
features, embedding model architecture, time discretization, reward function,
and random seeds. We find that varying these settings can significantly impact
learned policies, which suggests a need for caution when interpreting RL agent
output.
