Infomax strategies for an optimal balance between exploration and
  exploitation
  Proper balance between exploitation and exploration is what makes good
decisions, which achieve high rewards like payoff or evolutionary fitness. The
Infomax principle postulates that maximization of information directs the
function of diverse systems, from living systems to artificial neural networks.
While specific applications are successful, the validity of information as a
proxy for reward remains unclear. Here, we consider the multi-armed bandit
decision problem, which features arms (slot-machines) of unknown probabilities
of success and a player trying to maximize cumulative payoff by choosing the
sequence of arms to play. We show that an Infomax strategy (Info-p) which
optimally gathers information on the highest mean reward among the arms
saturates known optimal bounds and compares favorably to existing policies. The
highest mean reward considered by Info-p is not the quantity actually needed
for the choice of the arm to play, yet it allows for optimal tradeoffs between
exploration and exploitation.
