A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement
  Learning
  Prescribing optimal operation based on the condition of the system and,
thereby, potentially prolonging the remaining useful lifetime has a large
potential for actively managing the availability, maintenance and costs of
complex systems. Reinforcement learning (RL) algorithms are particularly
suitable for this type of problems given their learning capabilities. A special
case of a prescriptive operation is the power allocation task, which can be
considered as a sequential allocation problem, where the action space is
bounded by a simplex constraint. A general continuous action-space solution of
such sequential allocation problems has still remained an open research
question for RL algorithms. In continuous action-space, the standard Gaussian
policy applied in reinforcement learning does not support simplex constraints,
while the Gaussian-softmax policy introduces a bias during training. In this
work, we propose the Dirichlet policy for continuous allocation tasks and
analyze the bias and variance of its policy gradients. We demonstrate that the
Dirichlet policy is bias-free and provides significantly faster convergence,
better performance and better hyperparameters robustness over the
Gaussian-softmax policy. Moreover, we demonstrate the applicability of the
proposed algorithm on a prescriptive operation case, where we propose the
Dirichlet power allocation policy and evaluate the performance on a case study
of a set of multiple lithium-ion (Li-I) battery systems. The experimental
results show the potential to prescribe optimal operation, improve the
efficiency and sustainability of multi-power source systems.
