Automatic Exploration Process Adjustment for Safe Reinforcement Learning
  with Joint Chance Constraint Satisfaction
  In reinforcement learning (RL) algorithms, exploratory control inputs are
used during learning to acquire knowledge for decision making and control,
while the true dynamics of a controlled object is unknown. However, this
exploring property sometimes causes undesired situations by violating
constraints regarding the state of the controlled object. In this paper, we
propose an automatic exploration process adjustment method for safe RL in
continuous state and action spaces utilizing a linear nominal model of the
controlled object. Specifically, our proposed method automatically selects
whether the exploratory input is used or not at each time depending on the
state and its predicted value as well as adjusts the variance-covariance matrix
used in the Gaussian policy for exploration. We also show that our exploration
process adjustment method theoretically guarantees the satisfaction of the
constraints with the pre-specified probability, that is, the satisfaction of a
joint chance constraint at every time. Finally, we illustrate the validity and
the effectiveness of our method through numerical simulation.
