Deep reinforcement learning under signal temporal logic constraints
  using Lagrangian relaxation
  Deep reinforcement learning (DRL) has attracted much attention as an approach
to solve sequential decision making problems without mathematical models of
systems or environments. In general, a constraint may be imposed on a decision
making. In this study, we consider the optimal decision making problems with
constraints to complete temporal high-level tasks in the continuous
state-action domain. We describe the constraints using signal temporal logic
(STL), which is useful for time sensitive control tasks since it can specify
continuous signals within a bounded time interval. To deal with the STL
constraints, we introduce an extended constrained Markov decision process
(CMDP), which is called a $\tau$-CMDP. We formulate the STL constrained optimal
decision making problem as the $\tau$-CMDP and propose a two-phase constrained
DRL algorithm using the Lagrangian relaxation method. Through simulations, we
also demonstrate the learning performance of the proposed algorithm.
