Reinforcement Learning for Joint Optimization of Multiple Rewards
  Reinforcement Learning (RL) algorithms such as DQN owe their success to
Markov Decision Processes, and the fact that maximizing the sum of rewards
allows using backward induction and reduce to the Bellman optimality equation.
However, many real-world problems require optimization of an objective that is
non-linear in cumulative rewards for which dynamic programming cannot be
applied directly. For example, in a resource allocation problem, one of the
objectives is to maximize long-term fairness among the users. We notice that
when the function of the sum of rewards is considered, the problem loses its
Markov nature. This paper addresses and formalizes the problem of optimizing a
non-linear function of the long term average of rewards. We propose model-based
and model-free algorithms to learn the policy, where the model-based policy is
shown to achieve a regret of $\Tilde{O}\left(KDSA\sqrt{\frac{A}{T}}\right)$ for
$K$ users. Further, using the fairness in cellular base-station scheduling, and
queueing system scheduling as examples, the proposed algorithm is shown to
significantly outperform the conventional RL approaches.
