Closed-Form Analytical Results for Maximum Entropy Reinforcement
  Learning
  We introduce a mapping between Maximum Entropy Reinforcement Learning (MaxEnt
RL) and Markovian processes conditioned on rare events. In the long time limit,
this mapping allows us to derive analytical expressions for the optimal policy,
dynamics and initial state distributions for the general case of stochastic
dynamics in MaxEnt RL. We find that soft-$\mathcal{Q}$ functions in MaxEnt RL
can be obtained from the Perron-Frobenius eigenvalue and the corresponding left
eigenvector of a regular, non-negative matrix derived from the underlying
Markov Decision Process (MDP). The results derived lead to novel algorithms for
model-based and model-free MaxEnt RL, which we validate by numerical
simulations. The mapping established in this work opens further avenues for the
application of novel analytical and computational approaches to problems in
MaxEnt RL. We make our code available at:
https://github.com/argearriojas/maxent-rl-mdp-scripts
