Agent Incentives: A Causal Perspective
  We present a framework for analysing agent incentives using causal influence
diagrams. We establish that a well-known criterion for value of information is
complete. We propose a new graphical criterion for value of control,
establishing its soundness and completeness. We also introduce two new concepts
for incentive analysis: response incentives indicate which changes in the
environment affect an optimal decision, while instrumental control incentives
establish whether an agent can influence its utility via a variable X. For both
new concepts, we provide sound and complete graphical criteria. We show by
example how these results can help with evaluating the safety and fairness of
an AI system.
