Axioms for Rational Reinforcement Learning
  We provide a formal, simple and intuitive theory of rational decision making
including sequential decisions that affect the environment. The theory has a
geometric flavor, which makes the arguments easy to visualize and understand.
Our theory is for complete decision makers, which means that they have a
complete set of preferences. Our main result shows that a complete rational
decision maker implicitly has a probabilistic model of the environment. We have
a countable version of this result that brings light on the issue of countable
vs finite additivity by showing how it depends on the geometry of the space
which we have preferences over. This is achieved through fruitfully connecting
rationality with the Hahn-Banach Theorem. The theory presented here can be
viewed as a formalization and extension of the betting odds approach to
probability of Ramsey and De Finetti.
