Analyzing Micro-Founded General Equilibrium Models with Many Agents
  using Deep Reinforcement Learning
  Real economies can be modeled as a sequential imperfect-information game with
many heterogeneous agents, such as consumers, firms, and governments. Dynamic
general equilibrium (DGE) models are often used for macroeconomic analysis in
this setting. However, finding general equilibria is challenging using existing
theoretical or computational methods, especially when using microfoundations to
model individual agents. Here, we show how to use deep multi-agent
reinforcement learning (MARL) to find $\epsilon$-meta-equilibria over agent
types in microfounded DGE models. Whereas standard MARL fails to learn
non-trivial solutions, our structured learning curricula enable stable
convergence to meaningful solutions. Conceptually, our approach is more
flexible and does not need unrealistic assumptions, e.g., continuous market
clearing, that are commonly used for analytical tractability. Furthermore, our
end-to-end GPU implementation enables fast real-time convergence with a large
number of RL economic agents. We showcase our approach in open and closed
real-business-cycle (RBC) models with 100 worker-consumers, 10 firms, and a
social planner who taxes and redistributes. We validate the learned solutions
are $\epsilon$-meta-equilibria through best-response analyses, show that they
align with economic intuitions, and show our approach can learn a spectrum of
qualitatively distinct $\epsilon$-meta-equilibria in open RBC models. As such,
we show that hardware-accelerated MARL is a promising framework for modeling
the complexity of economies based on microfoundations.
