Managing engineering systems with large state and action spaces through
  deep reinforcement learning
  Decision-making for engineering systems can be efficiently formulated as a
Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). Typical
MDP and POMDP solution procedures utilize offline knowledge about the
environment and provide detailed policies for relatively small systems with
tractable state and action spaces. However, in large multi-component systems
the sizes of these spaces easily explode, as system states and actions scale
exponentially with the number of components, whereas environment dynamics are
difficult to be described in explicit forms for the entire system and may only
be accessible through numerical simulators. In this work, to address these
issues, an integrated Deep Reinforcement Learning (DRL) framework is
introduced. The Deep Centralized Multi-agent Actor Critic (DCMAC) is developed,
an off-policy actor-critic DRL approach, providing efficient life-cycle
policies for large multi-component systems operating in high-dimensional
spaces. Apart from deep function approximations that parametrize large state
spaces, DCMAC also adopts a factorized representation of the system actions,
being able to designate individualized component- and subsystem-level
decisions, while maintaining a centralized value function for the entire
system. DCMAC compares well against Deep Q-Network (DQN) solutions and exact
policies, where applicable, and outperforms optimized baselines that are based
on time-based, condition-based and periodic policies.
