Deep reinforcement learning driven inspection and maintenance planning
  under incomplete information and constraints
  Determination of inspection and maintenance policies for minimizing long-term
risks and costs in deteriorating engineering environments constitutes a complex
optimization problem. Major computational challenges include the (i) curse of
dimensionality, due to exponential scaling of state/action set cardinalities
with the number of components; (ii) curse of history, related to exponentially
growing decision-trees with the number of decision-steps; (iii) presence of
state uncertainties, induced by inherent environment stochasticity and
variability of inspection/monitoring measurements; (iv) presence of
constraints, pertaining to stochastic long-term limitations, due to resource
scarcity and other infeasible/undesirable system responses. In this work, these
challenges are addressed within a joint framework of constrained Partially
Observable Markov Decision Processes (POMDP) and multi-agent Deep Reinforcement
Learning (DRL). POMDPs optimally tackle (ii)-(iii), combining stochastic
dynamic programming with Bayesian inference principles. Multi-agent DRL
addresses (i), through deep function parametrizations and decentralized control
assumptions. Challenge (iv) is herein handled through proper state augmentation
and Lagrangian relaxation, with emphasis on life-cycle risk-based constraints
and budget limitations. The underlying algorithmic steps are provided, and the
proposed framework is found to outperform well-established policy baselines and
facilitate adept prescription of inspection and intervention actions, in cases
where decisions must be made in the most resource- and risk-aware manner.
