Importance of Environment Design in Reinforcement Learning: A Study of a
  Robotic Environment
  An in-depth understanding of the particular environment is crucial in
reinforcement learning (RL). To address this challenge, the decision-making
process of a mobile collaborative robotic assistant modeled by the Markov
decision process (MDP) framework is studied in this paper. The optimal
state-action combinations of the MDP are calculated with the non-linear Bellman
optimality equations. This system of equations can be solved with relative ease
by the computational power of Wolfram Mathematica, where the obtained optimal
action-values point to the optimal policy. Unlike other RL algorithms, this
methodology does not approximate the optimal behavior, it gives the exact,
explicit solution, which provides a strong foundation for our study. With this,
we offer new insights into understanding the action selection mechanisms in RL
by presenting various small modifications on the very same schema that lead to
different optimal policies.
