The AI Economist: Optimal Economic Policy Design via Two-level Deep
  Reinforcement Learning
  AI and reinforcement learning (RL) have improved many areas, but are not yet
widely adopted in economic policy design, mechanism design, or economics at
large. At the same time, current economic methodology is limited by a lack of
counterfactual data, simplistic behavioral models, and limited opportunities to
experiment with policies and evaluate behavioral responses. Here we show that
machine-learning-based economic simulation is a powerful policy and mechanism
design framework to overcome these limitations. The AI Economist is a
two-level, deep RL framework that trains both agents and a social planner who
co-adapt, providing a tractable solution to the highly unstable and novel
two-level RL challenge. From a simple specification of an economy, we learn
rational agent behaviors that adapt to learned planner policies and vice versa.
We demonstrate the efficacy of the AI Economist on the problem of optimal
taxation. In simple one-step economies, the AI Economist recovers the optimal
tax policy of economic theory. In complex, dynamic economies, the AI Economist
substantially improves both utilitarian social welfare and the trade-off
between equality and productivity over baselines. It does so despite emergent
tax-gaming strategies, while accounting for agent interactions and behavioral
change more accurately than economic theory. These results demonstrate for the
first time that two-level, deep RL can be used for understanding and as a
complement to theory for economic design, unlocking a new computational
learning-based approach to understanding economic policy.
