Opportunities of Hybrid Model-based Reinforcement Learning for Cell
  Therapy Manufacturing Process Control
  Driven by the key challenges of cell therapy manufacturing, including high
complexity, high uncertainty, and very limited process observations, we propose
a hybrid model-based reinforcement learning (RL) to efficiently guide process
control. We first create a probabilistic knowledge graph (KG) hybrid model
characterizing the risk- and science-based understanding of biomanufacturing
process mechanisms and quantifying inherent stochasticity, e.g., batch-to-batch
variation. It can capture the key features, including nonlinear reactions,
nonstationary dynamics, and partially observed state. This hybrid model can
leverage existing mechanistic models and facilitate learning from heterogeneous
process data. A computational sampling approach is used to generate posterior
samples quantifying model uncertainty. Then, we introduce hybrid model-based
Bayesian RL, accounting for both inherent stochasticity and model uncertainty,
to guide optimal, robust, and interpretable dynamic decision making. Cell
therapy manufacturing examples are used to empirically demonstrate that the
proposed framework can outperform the classical deterministic mechanistic model
assisted process optimization.
