RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement
  Pathway Optimization
  Joint replacement is the most common inpatient surgical treatment in the US.
We investigate the clinical pathway optimization for knee replacement, which is
a sequential decision process from onset to recovery. Based on episodic claims
from previous cases, we view the pathway optimization as an intelligence
crowdsourcing problem and learn the optimal decision policy from data by
imitating the best expert at every intermediate state. We develop a
reinforcement learning-based pipeline that uses value iteration, state
compression and aggregation learning, kernel representation and cross
validation to predict the best treatment policy. It also provides forecast of
the clinical pathway under the optimized policy. Empirical validation shows
that the optimized policy reduces the overall cost by 7 percent and reduces the
excessive cost premium by 33 percent.
