Preference Communication in Multi-Objective Normal-Form Games
  We consider preference communication in two-player multi-objective
normal-form games. In such games, the payoffs resulting from joint actions are
vector-valued. Taking a utility-based approach, we assume there exists a
utility function for each player which maps vectors to scalar utilities and
consider agents that aim to maximise the utility of expected payoff vectors. As
agents typically do not know their opponent's utility function or strategy,
they must learn policies to interact with each other. Inspired by Stackelberg
games, we introduce four novel preference communication protocols to aid agents
in arriving at adequate solutions. Each protocol describes a specific approach
for one agent to communicate preferences over their actions and how another
agent responds. Additionally, to study when communication emerges, we introduce
a communication protocol where agents must learn when to communicate. These
protocols are subsequently evaluated on a set of five benchmark games against
baseline agents that do not communicate. We find that preference communication
can alter the learning process and lead to the emergence of cyclic policies
which had not been previously observed in this setting. We further observe that
the resulting policies can heavily depend on the characteristics of the game
that is played. Lastly, we find that communication naturally emerges in both
cooperative and self-interested settings.
