The Good Shepherd: An Oracle Agent for Mechanism Design
  From social networks to traffic routing, artificial learning agents are
playing a central role in modern institutions. We must therefore understand how
to leverage these systems to foster outcomes and behaviors that align with our
own values and aspirations. While multiagent learning has received considerable
attention in recent years, artificial agents have been primarily evaluated when
interacting with fixed, non-learning co-players. While this evaluation scheme
has merit, it fails to capture the dynamics faced by institutions that must
deal with adaptive and continually learning constituents. Here we address this
limitation, and construct agents ("mechanisms") that perform well when
evaluated over the learning trajectory of their adaptive co-players
("participants"). The algorithm we propose consists of two nested learning
loops: an inner loop where participants learn to best respond to fixed
mechanisms; and an outer loop where the mechanism agent updates its policy
based on experience. We report the performance of our mechanism agents when
paired with both artificial learning agents and humans as co-players. Our
results show that our mechanisms are able to shepherd the participants
strategies towards favorable outcomes, indicating a path for modern
institutions to effectively and automatically influence the strategies and
behaviors of their constituents.
