Lifted Model Checking for Relational MDPs
  Probabilistic model checking has been developed for verifying systems that
have stochastic and nondeterministic behavior. Given a probabilistic system, a
probabilistic model checker takes a property and checks whether or not the
property holds in that system. For this reason, probabilistic model checking
provide rigorous guarantees. So far, however, probabilistic model checking has
focused on propositional models where a state is represented by a symbol. On
the other hand, it is commonly required to make relational abstractions in
planning and reinforcement learning. Various frameworks handle relational
domains, for instance, STRIPS planning and relational Markov Decision
Processes. Using propositional model checking in relational settings requires
one to ground the model, which leads to the well known state explosion problem
and intractability. We present pCTL-REBEL, a lifted model checking approach for
verifying pCTL properties of relational MDPs. It extends REBEL, a relational
model-based reinforcement learning technique, toward relational pCTL model
checking. PCTL-REBEL is lifted, which means that rather than grounding, the
model exploits symmetries to reason about a group of objects as a whole at the
relational level. Theoretically, we show that pCTL model checking is decidable
for relational MDPs that have a possibly infinite domain, provided that the
states have a bounded size. Practically, we contribute algorithms and an
implementation of lifted relational model checking, and we show that the lifted
approach improves the scalability of the model checking approach.
