Optimal sequential decision making with probabilistic digital twins
  Digital twins are emerging in many industries, typically consisting of
simulation models and data associated with a specific physical system. One of
the main reasons for developing a digital twin, is to enable the simulation of
possible consequences of a given action, without the need to interfere with the
physical system itself. Physical systems of interest, and the environments they
operate in, do not always behave deterministically. Moreover, information about
the system and its environment is typically incomplete or imperfect.
Probabilistic representations of systems and environments may therefore be
called for, especially to support decisions in application areas where actions
may have severe consequences.
  In this paper we introduce the probabilistic digital twin (PDT). We will
start by discussing how epistemic uncertainty can be treated using measure
theory, by modelling epistemic information via $\sigma$-algebras. Based on
this, we give a formal definition of how epistemic uncertainty can be updated
in a PDT. We then study the problem of optimal sequential decision making. That
is, we consider the case where the outcome of each decision may inform the
next. Within the PDT framework, we formulate this optimization problem. We
discuss how this problem may be solved (at least in theory) via the maximum
principle method or the dynamic programming principle. However, due to the
curse of dimensionality, these methods are often not tractable in practice. To
mend this, we propose a generic approximate solution using deep reinforcement
learning together with neural networks defined on sets. We illustrate the
method on a practical problem, considering optimal information gathering for
the estimation of a failure probability.
