Bounded Risk-Sensitive Markov Games: Forward Policy Design and Inverse
  Reward Learning with Iterative Reasoning and Cumulative Prospect Theory
  Classical game-theoretic approaches for multi-agent systems in both the
forward policy design problem and the inverse reward learning problem often
make strong rationality assumptions: agents perfectly maximize expected
utilities under uncertainties. Such assumptions, however, substantially
mismatch with observed humans' behaviors such as satisficing with sub-optimal,
risk-seeking, and loss-aversion decisions. In this paper, we investigate the
problem of bounded risk-sensitive Markov Game (BRSMG) and its inverse reward
learning problem for modeling human realistic behaviors and learning human
behavioral models. Drawing on iterative reasoning models and cumulative
prospect theory, we embrace that humans have bounded intelligence and maximize
risk-sensitive utilities in BRSMGs. Convergence analysis for both the forward
policy design and the inverse reward learning problems are established under
the BRSMG framework. We validate the proposed forward policy design and inverse
reward learning algorithms in a navigation scenario. The results show that the
behaviors of agents demonstrate both risk-averse and risk-seeking
characteristics. Moreover, in the inverse reward learning task, the proposed
bounded risk-sensitive inverse learning algorithm outperforms a baseline
risk-neutral inverse learning algorithm by effectively recovering not only more
accurate reward values but also the intelligence levels and the risk-measure
parameters given demonstrations of agents' interactive behaviors.
