Exploiting Expert-guided Symmetry Detection in Offline Reinforcement
  Learning
  Offline estimation of the dynamical model of a Markov Decision Process (MDP)
is a non-trivial task that greatly depends on the data available to the
learning phase. Sometimes the dynamics of the model is invariant with respect
to some transformations of the current state and action. Recent works showed
that an expert-guided pipeline relying on Density Estimation methods as Deep
Neural Network based Normalizing Flows effectively detects this structure in
deterministic environments, both categorical and continuous-valued. The
acquired knowledge can be exploited to augment the original data set, leading
eventually to a reduction in the distributional shift between the true and the
learnt model. Such data augmentation technique can be exploited as a
preliminary process to be executed before the adoption of an Offline
Reinforcement Learning architecture, increasing its performance. In this work
we extend the paradigm to also tackle non deterministic MDPs, in particular 1)
we propose a detection threshold in categorical environments based on
statistical distances, and 2) we show that the former results lead to a
performance improvement when solving the learnt MDP and then applying the
optimal policy in the real environment.
