Green Simulation Assisted Reinforcement Learning with Model Risk for
  Biomanufacturing Learning and Control
  Biopharmaceutical manufacturing faces critical challenges, including
complexity, high variability, lengthy lead time, and limited historical data
and knowledge of the underlying system stochastic process. To address these
challenges, we propose a green simulation assisted model-based reinforcement
learning to support process online learning and guide dynamic decision making.
Basically, the process model risk is quantified by the posterior distribution.
At any given policy, we predict the expected system response with prediction
risk accounting for both inherent stochastic uncertainty and model risk. Then,
we propose green simulation assisted reinforcement learning and derive the
mixture proposal distribution of decision process and likelihood ratio based
metamodel for the policy gradient, which can selectively reuse process
trajectory outputs collected from previous experiments to increase the
simulation data-efficiency, improve the policy gradient estimation accuracy,
and speed up the search for the optimal policy. Our numerical study indicates
that the proposed approach demonstrates the promising performance.
